How to use HTTPClient for a multipart/form-data upload?

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By Karen S.

In out Godot 3.1 project, we are trying to use the HTTPClient class to upload a file to a server. (A self-hosted Seafile instance, in this case).

As an initial test, we just send a string (“test test test test”) as a text file.
The parent_dir and relative_path form fields are expected by Seafile.

The code is largely copied from this tutorial.

extends Node2D

# HTTPClient demo
# This simple class can do HTTP requests; it will not block, but it needs to be polled.

func _init():
  var err = 0
  var http = HTTPClient.new() # Create the Client.

  err = http.connect_to_host(SEAFILE_URL, 443, true) # Connect to host/port.
  assert(err == OK) # Make sure connection was OK.

  # Wait until resolved and connected.
  while http.get_status() == HTTPClient.STATUS_CONNECTING or http.get_status() == HTTPClient.STATUS_RESOLVING:
        http.poll()
        print("Connecting...")
        OS.delay_msec(500)

    assert(http.get_status() == HTTPClient.STATUS_CONNECTED) # Could not connect


    var r = PoolByteArray()

  r.append("------WebKitFormBoundary7MA4YWxkTrZu0gW\r\n")
  r.append("Content-Disposition: form-data; name=\"file\"; filename=\"test123.txt\"\r\n")
  r.append("Content-Type: text/plain\r\n\r\n")
  r.append("test test test test")
  r.append("------WebKitFormBoundary7MA4YWxkTrZu0gW\r\n")
  r.append("Content-Disposition: form-data; name=\"relative_path\"\r\n\r\n")
  r.append("SaveFiles")
  r.append("------WebKitFormBoundary7MA4YWxkTrZu0gW\r\n")   
  r.append("Content-Disposition: form-data; name=\"parent_dir\"\r\n\r\n")
  r.append("/")
  r.append("------WebKitFormBoundary7MA4YWxkTrZu0gW--")   




  # Some headers
  var headers = [
      "Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW; charset=utf-8",
      "Content-Length: " + str(r.size())
  ]

  print("sending: " + r.get_string_from_utf8())

  err = http.request_raw(HTTPClient.METHOD_POST, "/seafhttp/upload-aj/TOKEN", headers, r) # Request a page from the site (this one was chunked..)
  assert(err == OK) # Make sure all is OK.

  while http.get_status() == HTTPClient.STATUS_REQUESTING:
      # Keep polling for as long as the request is being processed.
      http.poll()
      print("Requesting...")
      if not OS.has_feature("web"):
          OS.delay_msec(500)
      else:
          # Synchronous HTTP requests are not supported on the web,
          # so wait for the next main loop iteration.
          yield(Engine.get_main_loop(), "idle_frame")

  assert(http.get_status() == HTTPClient.STATUS_BODY or http.get_status() == HTTPClient.STATUS_CONNECTED) # Make sure request finished well.

  print("response? ", http.has_response()) # Site might not have a response.

  if http.has_response():
      # If there is a response...

      headers = http.get_response_headers_as_dictionary() # Get response headers.
      print("code: ", http.get_response_code()) # Show response code.
      print("**headers:\\n", headers) # Show headers.

     # Getting the HTTP Body

      if http.is_response_chunked():
          # Does it use chunks?
          print("Response is Chunked!")
      else:
          # Or just plain Content-Length
          var bl = http.get_response_body_length()
          print("Response Length: ",bl)

      # This method works for both anyway

      var rb = PoolByteArray() # Array that will hold the data.

      while http.get_status() == HTTPClient.STATUS_BODY:
          # While there is body left to be read
          http.poll()
          var chunk = http.read_response_body_chunk() # Get a chunk.
          if chunk.size() == 0:
              # Got nothing, wait for buffers to fill a bit.
              OS.delay_usec(1000)
          else:
              rb = rb + chunk # Append to read buffer.

      # Done!

      print("bytes got: ", rb.size())
      var text = rb.get_string_from_ascii()
      print("Text: ", text)

Apparently, the file is not send correctly to the server, because the server always responds: {“error”: “No file.”}

Conversely, this Java/OkHttp code here does essentially the same thing, and the request succeeds, the file is uploaded to the server successfully:

public static void main(String[] args) throws IOException {
    HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
    logging.setLevel(HttpLoggingInterceptor.Level.BODY);
    OkHttpClient client = new OkHttpClient.Builder()
            .addInterceptor(logging)
            .build();

    MediaType mediaType = MediaType.parse("multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW");
    RequestBody body = RequestBody.create(mediaType,
            "------WebKitFormBoundary7MA4YWxkTrZu0gW" +
                    "\r\nContent-Disposition: form-data; name=\"file\"; fileName=\"test123.txt\"" +
                    "\r\nContent-Type: text/plain" +
                    "\r\n\r\ntest test test test\r\n" +
                    "------WebKitFormBoundary7MA4YWxkTrZu0gW" +
                    "\r\nContent-Disposition: form-data; name=\"relative_path\"" +
                    "\r\n\r\nSaveFiles\r\n" +
                    "------WebKitFormBoundary7MA4YWxkTrZu0gW" +
                    "\r\nContent-Disposition: form-data; name=\"parent_dir\"" +
                    "\r\n\r\n/\r\n" +
                    "------WebKitFormBoundary7MA4YWxkTrZu0gW--");
    okhttp3.Request request = new Request.Builder()
            .url("SEAFILE_URL/seafhttp/upload-aj/TOKEN")
            .post(body)
            .addHeader("content-type", "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW")
            .build();

    Response response = client.newCall(request).execute();
    System.out.println(response.toString());
}

Does anybody know what we’re doing wrong?

I’m currently facing the same issue, did you manage to solve this?

chris.brkt | 2022-03-17 00:33

Hi Chris, no, I’m afraid not.

Karen S. | 2022-03-17 08:18

:bust_in_silhouette: Reply From: chris.brkt

I managed to finally find a solution for this. It’s basically issues in the syntax.
I elaborated more on it on my question here Uploading file to server through REST - Archive - Godot Forum.

Maybe in your case, you should go on the next line after each body value and before the boundary? In Java, there’s \r\n after SaveFiles for example which seems to be missing in your gdscript. The same for the parent_dir value.
And maybe, just in case, encode each line to utf8?
So I would try something like:

  var r = PoolByteArray()

  r.append_array("--BodyBoundaryHere".to_utf8())
  r.append_array("\r\nContent-Disposition: form-data; name=\"file\"; filename=\"test123.txt\"\r\n".to_utf8())
  r.append_array("Content-Type: text/plain\r\n".to_utf8())
  r.append_array("\r\n".to_utf8())
  r.append_array("test test test test".to_utf8())
  r.append_array("\r\n".to_utf8())
  r.append_array("--BodyBoundaryHere".to_utf8())
  r.append_array("\r\nContent-Disposition: form-data; name=\"relative_path\"\r\n".to_utf8())
  r.append_array("\r\n".to_utf8())
  r.append_array("SaveFiles".to_utf8())
  r.append_array("\r\n".to_utf8())
  r.append_array("--BodyBoundaryHere".to_utf8())
  r.append_array("\r\nContent-Disposition: form-data; name=\"parent_dir\"\r\n".to_utf8())
  r.append_array("\r\n".to_utf8())
  r.append_array("/".to_utf8())
  r.append_array("\r\n".to_utf8())
  r.append_array("--BodyBoundaryHere--".to_utf8())

  # Some headers
  var headers = [
      "Content-Type: multipart/form-data; boundary=\"BodyBoundaryHere\""",
      "Content-Length: " + str(r.size())
  ]

I didn’t compile it, it’s just something that I can think of on top of my head.