0 votes

I am trying to make a Mandarin learning game for my kids. They hate writing the Chinese characters, so I'm thinking whether I can make a game where the player writes Chinese characters, and if their writing matches (within a range of accuracy) the actual writing, then they can (i dunno) kill the monsters.

How could I go about doing something like this?

For example, is it possible to train (as in machine training kind of train) the game what the acceptable handwritings are for each character, and then use the model trained to score any writing by the player to give an accuracy, and if the accuracy is above 70%, then give green light to kill the monster?

in Engine by (15 points)

I don't think you can do this with Godot.But you can use python opencv library to recognize characters.You can make some server on python which gets data(images) from your godot application ,then sends back the result.

Hi
I can not help you on the godot side, but if you are new to NeuralNetworks the following links might help. If you already know NeuralNetworks you can skip my comment.

As general introduction I found 3Blue1Brown on youtube rather helpfull:
https://www.youtube.com/watch?v=aircAruvnKk

Codelabs lets you use external hardware (you need a gpu) and gives you hands on tutorials for a betters start.
https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist#0

Maybe you will find a pretrained network for your problem, but if not there you can design, train and export a new network. For the training you will need a big amount of labeled data, in your case lots of pictures of Chinese characters with notes what character this is. I am sure you will find such a database if you search for one on the internet.

1 Answer

+1 vote
Best answer

Handwriting Gestures

First break your game into 3 groups

  • Gesture Recognition

  • Character Identification

    • Either OCR
    • Compare Images using RMS
    • Compare hasables
  • Reward System

    • Totally up to you

Happened across this post and remembered your question since i liked it and was able to make exactly what you wanted by modifying that script and compiling the tesseract-lib using GDNative even for Android as well, but know that there are also pre-build binaries you can use.

I had alot of fun doing this so i feel like spoon feeding you would be robing you of that Joy and i also can't remember all the steps i took.

P.S. Use Line2D instead to draw you get much better results

by (4,982 points)
selected by

My version of the ChalkBoardApp i use the Thread for character recognizing functions which are excluded here

extends ColorRect

export var pen_color = Color(1, 1, 0)
export(float, 1, 50) var pen_size = 5
export(float, 1, 10) var pen_delay = 1

onready var viewport = Viewport.new()
onready var board = TextureRect.new()
onready var pen = Line2D.new()
onready var thread = Thread.new()
onready var thread_timer = Timer.new()

var mouse_pos = Vector2()
var tex : Image

func _ready():
    var rt = get_rect().size

    viewport.set_size(rt)
    viewport.set_usage(Viewport.USAGE_2D)
    viewport.set_clear_mode(Viewport.CLEAR_MODE_ONLY_NEXT_FRAME)
    viewport.set_transparent_background(true)
    viewport.set_vflip(true)

    pen.set_joint_mode(Line2D.LINE_JOINT_ROUND)
    pen.set_begin_cap_mode(Line2D.LINE_CAP_ROUND)
    pen.set_end_cap_mode(Line2D.LINE_CAP_ROUND)
    pen.set_default_color(pen_color)
    pen.set_antialiased(true)
    pen.set_width(pen_size)

    # Use a sprite to display the result texture
    board.set_texture(viewport.get_texture())

    add_child(viewport)
    add_child(board)
    add_child(thread_timer)
    viewport.add_child(pen)

func _input(event):
    if event is InputEventScreenTouch:
        if not event.pressed:
            mouse_pos = Vector2()
            pen.clear_points()
        else:
            mouse_pos = get_local_mouse_position()
    if event is InputEventScreenDrag:
        if event.relative.length() > pen_delay:
            mouse_pos = get_local_mouse_position()
            pen.add_point(mouse_pos)
Welcome to Godot Engine Q&A, where you can ask questions and receive answers from other members of the community.

Please make sure to read How to use this Q&A? before posting your first questions.
Social login is currently unavailable. If you've previously logged in with a Facebook or GitHub account, use the I forgot my password link in the login box to set a password for your account. If you still can't access your account, send an email to webmaster@godotengine.org with your username.