Is it possible to make the audio data in gdscript only?

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By SIsilicon

It’s awesome that you are able to make procedurally generated audio. But can it be done without gdnative.

:bust_in_silhouette: Reply From: Akorian

I don’t see a reason why it shouldn’t be possible.

For example you can create a pool byte array in your script and add the bytes that you generated manually to this buffer (make sure you’re using the right format). Then simply set is as .data like I did in the original answer:

var Sample = AudioStreamSample.new()
Sample.data = YOUR_GENERATED_POOL_BYTE_ARRAY # unsafe?
Sample.format = AudioStreamSample.FORMAT_16_BITS
Sample.loop_mode = AudioStreamSample.LOOP_DISABLED
Sample.loop_begin = 0
Sample.loop_end = 0
Sample.mix_rate = 44100 # you can also use AudioServer.get_mix_rate() to get the sampling rate of the current device
Sample.stereo = false

Keep in mind that doing this in GDScript will be slower than doing it in GDNative.

Edit: If you’re trying to generate fully procedual audio during playback it might not work very well.

You could try using Sample.data as a ring buffer (with looping enabled, obviously) and sample into it while playing but you’ll have to make sure that it is synchronized properly. Also you might need to use multithreading to prevent framedrops while generating new audio.

Edit2: here’s a quick and dirty example

func ShortToBytes(value): # value: 16 bit INTEGER! [0 - 65535]
	var bytes = PoolByteArray()
	var X = value / 255
	bytes.append(value - X - 128) # are godot bytes signed or unsigned?
	bytes.append(X - 128)
	return bytes
    
func GenerateAndPlay():
	# generate data
	var audio_buffer = PoolByteArray()
	var samples_per_second = AudioServer.get_mix_rate()
	var length = int(samples_per_second * 2.5) # length in number of samples (here 2.5 seconds)

	for i in range(0, length): # for number of samples
		var sample = int(sin(float(i) / samples_per_second * 440.0 * PI * 2.0) * 16000 + 16000) # generate a 440 Hz sine wave
	audio_buffer.append_array(ShortToBytes(sample))
	
	# create sample from generated data
	var sample = AudioStreamSample.new()
	sample.data = audio_buffer
	sample.format = AudioStreamSample.FORMAT_16_BITS
	sample.loop_mode = AudioStreamSample.LOOP_DISABLED
	sample.loop_begin = 0
	sample.loop_end = 0
	sample.mix_rate = samples_per_second
	sample.stereo = false
	
	# create audio stream player & play sample
	var asp = AudioStreamPlayer.new()
	asp.volume_db = -13.0 # don't blast your ears!
	asp.stream = sample # set the sample as source
	add_child(asp) # add stream player to the scene tree so the output is heard
	asp.play()

It works but I’m not sure if the value mapping is correct. Using Godot for this kind of stuff doesn’t feel very great, maybe in the future there will be more solid options for these kind of tasks.

Noice.
And interesting.

SIsilicon | 2018-05-19 18:06

I added a quick GDScript example that shows the general steps.

Akorian | 2018-05-19 18:16

I tried it.
It was actually pretty much instant.

I even started to create other waveforms.

SIsilicon | 2018-05-20 02:04

Kind of stuck on triangles.

Also I went into the docs and found this method called var2bytes. We should try that out.

SIsilicon | 2018-05-20 02:05

I’m not sure if var2bytes really helps since there is no ‘short’ type in godot. Shorts are very helpful because they represent 1 sample of the audio stream at 16 bit accuracy (for one channel).

Overall the performance was okay-ish for me. As a comparison (not actually measured), it took about the same time to generate a 15 second sine wave in GDScript that it took to load a complex 90 seconds pxtone file, process it in GDNative and pass it back to GDScript.

Akorian | 2018-05-20 08:04

I honestly really wanted to work with gdnative.
I already know how to program in c++, which I know is only one of languages you can use.
But I’m still confused by the process of converting it into a gdnative library. Especially since I’m using a Windows machine. The examples didn’t have that library version.

SIsilicon | 2018-05-20 15:35

The major difference in using GDNative compared to a normal .dll file is that you have to define some functions within your C/C++ code for communicating / converting variables between the library and GDScript. It’s a bit like a wrapper. From there you can do anything you want. There’s also a bit of additional coding on the GDScript side, though you can keep that very minimal.

However, I agree that it’s a bit confusing. I’m using the GDNative C interface (though internally my code is written in C++) - mostly because I couldn’t get the C++ bindings to work.
Setting up the interface in C seems very cumbersome if I compare it to the C++ examples in one of the tutorials: Home | Ravone Arts

I hope there will be a few thorough GDNative tutorials in the future.

Akorian | 2018-05-20 16:12

Ram it! The tutorial doesn’t talk about compiling in Windows. Doesn’t anyone like windows? :cry:

SIsilicon | 2018-05-20 17:37

:bust_in_silhouette: Reply From: Zylann

My contribution to read and write mono 8-bit and 16-bit audio samples: How to read audio samples as [-1..1] floats? - Archive - Godot Forum