What's it all about, Alfie?

Today I wanted to get a simplified File Upload scheme working on an app, where the backend was a Clojure app, and the front-end was simple HTML/jQuery - nothing fancy. I knew it had to be multi-part MIME REST calls, but I needed to work out the details about extracting the file, as well as the JSON metadata from the call.

Let's start with how it's shipped up to the server. We have a Bootstrap-based front-end, and we wanted to allow for the user to enter the Description of the file, and then browse their local filesystem for the file to upload. It's about as fast and efficient as I can imagine, so we had the following:

  <div class="row" align="center" style="margin: 5px 0px 0px  0px;">
    <div class="col-sm-1" style="margin-top:5px;">
    </div>
    <div class="col-sm-10" style="margin-top:5px;">
      <div id="addDocsDiv" style="margin: 0px 0px 0px 0px;">
        <form id="post_site_doc">
          <div class="input-group">
            <label class="control-label col-sm-2"
              style="margin-top: 7px; padding-right: 0px;"
              align="right">Description:</label>
            <div class="col-sm-10">
              <input id="doc_desc" class="form-control" type="text"
                aria-label="Description of File"/>
            </div>
            <span class="input-group-btn">
              <label for="site_file" class="btn btn-default">Browse</label>
              <input id="site_file" type="file" style="visibility:hidden;"
                onChange="sendSiteDoc(this);"/>
            </span>
          </div>
        </form>
      </div>
    </div>
    <div class="col-sm-1" style="margin-top:5px;">
    </div>
  </div>

this gets rendered as:

which is about as clean as I can make it.

The point is that there are two key components:

• The site_file file input tag with the onChange action of calling sendSiteDoc(this) - on line 17.
• The doc_desc text input tag - on line 12.

these are where we will be pulling the data to send to the service.

Then the sendSiteDoc() function looks like:

  /*
   * Function to take the 'this' from an 'input' file uploader, that was
   * triggered off the 'onChange' event, and will use the description field
   * on the page to send the whole thing up to the server, and then reload
   * the site from the server to make it all look nice.
   *
   * We are using fetch() as opposed to jQuery as it's just easier in this
   * case.
   */
  function sendSiteDoc(inp) {
    const email = sessionStorage.getItem('login');
    if (email) {
      const sid = $("#siteId").val();
      if (!isUuid(sid)) {
        let cont = '<div class="alert alert-danger" role="alert">';
        cont += '<strong>Sorry!</strong>';
        cont += " You have to be viewing a site to upload a document!</div>";
        $("#status").html(cont).show().fadeOut(5000);
        return null;
      }
      // pull the data from the form and make a FormData
      const file = inp.files[0];
      const description = $("#doc_desc").val();
      $("#doc_desc").val('');
      // ...we want to have the metadata as a second part of the multi-part
      let meta = {};
      if (nullIfEmpty(description)) {
        meta.description = description;
      }
      // now let's package this into a FormData() - two parts, please
      let fdata = new FormData();
      fdata.append("file", file);
      fdata.append("meta", JSON.stringify(meta));
      fetch('/sites/' + sid + '/doc', { method: 'POST', body: fdata })
        .then(resp => {
          if (!resp.ok) {
            let cont = '<div class="alert alert-danger" role="alert">';
            cont += '<strong>Sorry!</strong>';
            cont += " We could not upload this document to the system!</div>";
            $("#status").html(cont).show().fadeOut(10000);
            return null;
          }
          return resp.json();
        })
        .then(data => {
          if (data) {
            // ...and refresh the data on this page, as we're on the page...
            loadSite(sid);
            // ...and drop a nice status message
            console.log('success: ', data);
            let cont = '<div class="alert alert-success" role="alert">';
            cont += '<strong>Success!</strong>';
            cont += ' The document was uploaded to the service.</div>';
            $("#status").html(cont).show().fadeOut(5000);
          }
        })
        .catch(err => {
            let cont = '<div class="alert alert-danger" role="alert">';
            cont += '<strong>Sorry!</strong>';
            cont += " We had a problem while uploading this document to the system!</div>";
            $("#status").html(cont).show().fadeOut(10000);
        });
    } else {
      let cont = '<div class="alert alert-danger" role="alert">';
      cont += '<strong>Error!</strong>';
      cont += " You must be logged in to make changes!</div>";
      $("#status").html(cont).show().fadeOut(5000);
    }
  }

Where the key section is really lines 21-34:

      // pull the data from the form and make a FormData
      const file = inp.files[0];
      const description = $("#doc_desc").val();
      $("#doc_desc").val('');
      // ...we want to have the metadata as a second part of the multi-part
      let meta = {};
      if (nullIfEmpty(description)) {
        meta.description = description;
      }
      // now let's package this into a FormData() - two parts, please
      let fdata = new FormData();
      fdata.append("file", file);
      fdata.append("meta", JSON.stringify(meta));
      fetch('/sites/' + sid + '/doc', { method: 'POST', body: fdata })

and we take the first file from the input selection, and the description from that input field, and then create the meta Object to hold the description for sending to the server. Finally, we create a FormData and then append the two parts to it, making sure to stringify() the JSON to make it shippable to the server.

The rest of the code is about verifying that we are logged into the service, but it'll check that anyway, and then handling error messages and such, but the key lines are just the few, above.

So what does the Clojure back-end look like? Well... we are going to use the ring wrap-defaults as it includes so many of the useful middleware for ring, and the key inclusion here is the params handling:

  (def site-defaults*
    "Ring defaults allows us to package a lot of the standard middleware out
    there into one wrap-defaults step, and then we can configure it all here
    by simply augmenting the site-defaults 'starter pack'. This allows us to
    minimize the dependencies and control a lot of things very simply with this
    one augmentation map."
    (assoc site-defaults
      :cookies true
      :params {
        :keywordize true      ;; this converts the string keys to keywords
        :multipart true       ;; this handles the multi-part MIME
        :nested true
        :urlencoded true
      }
      :proxy true
      :security false
      :session {
        :flash true
        :store (carmine-store :bedrock)
      }
    )
  )
 
  (def app
    "The actual ring handler that is run -- this is the routes above
     wrapped in various middlewares."
    (let [backend (session-backend {:unauthorized-handler unauthorized-handler})]
      (-> app-routes
          (wrap-access-rules {:rules rules :on-error unauthorized-handler})
          (wrap-authorization backend)
          (wrap-authentication backend)
          wrap-user
          (wrap-json-body {:key-fn ->kebab-case-keyword})
          wrap-json-with-padding
          wrap-cors
          (wrap-defaults site-defaults*)   ;; this does the main work
          wrap-logging
          wrap-gzip)))

We then needed to have a route to handle the file upload, and we have a subset of the routes here:

  (defn sites-routes
    "These are the routes for the sites, and should all be placed under the
    '/sites' context off the main defroutes for the server. It's a simple
    way to isolate these endpoints here, and then not have to worry about them
    interferring with anything later."
    []
    (routes
      (GET "/" [:as {user :user}]
        (pull-sites user))
      (POST ["/:id/doc" :id uuid-pattern] [id :as {user :user params :params}]
        (post-site-doc user (->uuid id) (:file params) (parse-string (:meta params))))
      ))

where the POST is the key, and the multi-part FormData in Javascript will come into the call in the params where the keys in the params are the append()-ed names of the elements in the FormData.

As a final step, we need to parse the :meta value from JSON, and then we are good to go. The arguments to post-site-doc will then look like:

    {
      :filename "words.txt"
      :content-type "text/plain"
      :tempfile #object[java.io.File ...]
      :size 1021
    }

and:

    {
      :description "this is a file description"
    }

At this point, we can treat the :tempfile like any file because it is a simple temp file that will be retained for about an hour, and then dropped. There is nothing magic about it, and the only real question is having enough temp filesystem space to handle any upload, and that's a very simple thing to arrange.

With this, we can handle all kinds of file uploads, and the MIME type will be sent from the client so that we can retain that, if we want (and we do), and then be able to serve this file back up to the caller at any time. Even in a streaming format, if that's what they want.

It was really nice to get this all nailed down so easily. 🙂

This entry was posted on Friday, March 28th, 2025 at 10:06 am and is filed under Clojure Coding, Javascript Coding. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.