spork/http errors

[BillScherer]

I'm a long time Python programmer (28 years). I've been studying Emacs lisp and SBCl for a a few years now as well. Discovered Janet a couple of months ago, and it's fantastic. I am working on an image gallery to learn. I wrote it in Python first, and then in Janet. I got it working with circlet, but then while improving the code, I saw a thread that recommended using spork/http instead. So yesterday, I converted the circlet version to use spork/http, which was not difficult. It is working, but not 100% reliably. Sometimes it fails to delivery an image (whether that be a thumbnail, or the full size image). Whether it succeeds or fails, I always get the error below in the terminal where the server is running:

GET 200 /IMG?name=IMG_5437.JPG elapsed 2.987ms
error: expected integer key for buffer in range [0, 0), got :body
  in send-response [/usr/local/lib/janet/spork/http.janet] (tailcall) on line 328, column 24
  in server-handler [/usr/local/lib/janet/spork/http.janet] (tailcall) on line 414, column 3

My Janet version is 1.32.1-a10b4f61
I have updated all installed packages with "sudo jpm update-installed"; the error persists.
I get no errors with the circlet version.

Thanks for any insight into this error.

[sogaiu]

Not sure if the following will help, but FWIW, below are some observations having looked at spork/http a bit.

ATM line 328 in send-response (a line mentioned in the stack trace above) has the code:

  (write-body conn buf (in response :body)))

May be the error comes about because response is somehow an empty buffer when it should be a table or struct (or something else that when passed to in along with :body behaves appropriately).

Some evidence that supports this thinking:

$ janet
Janet 1.32.1-9142f38c linux/x64/gcc - '(doc)' for help
repl:1:> (in @"" :body)
error: expected integer key for buffer in range [0, 0), got :body
  in _thunk [repl] (tailcall) on line 1, column 1

Looking around for calls to send-response that might be relevant, I see these lines in server-handler:

    # Do something with request header
    (def response (handler req))

    # Now send back response
    (send-response conn response @"")))

If the last line is where the relevant call to send-response is made, perhaps response is being bound to a buffer as a result of the call to handler.

A bit further up in server-handler are these lines:

  [conn handler]
  (def handler (middleware handler))

and middleware has this definition:

(defn middleware
  "Coerce any type to http middleware"
  [x]
  (case (type x)
    :function x
    :number (let [msg (get status-messages x)]
              (assert x (string "unknown http status code when making middleware: " x))
              (fn mw [&] {:status x :body msg}))
    :string (bytes-to-mw x)
    :buffer (bytes-to-mw x)
    (fn mw [&] x)))

Looking at the case and definition of bytes-to-mw it seems to me that :function x may be relevant here -- i.e. none of the other cases seem like they apply. So middleware may be just returning its argument (a function) in this case.

So perhaps the handler argument passed to server-handler is a function which ends up returning an empty buffer at least sometimes?

[BillScherer]

Thanks for the info! I will try to digest it and see what I can change.

[BillScherer]

Below is my function for returning an image. I do not see how response could be empty.

(defn image [request]
  
  (def fname (query-value request "name"))

  (var fd (file/open fname :rb))
  (var data (file/read fd :all ))
  (file/close fd)
  
  (var response {:headers "Content-type: image/jpg"
		 :status 200
		 :body data})
  
  (http/send-response (get request :connection) response))

(note that (query-value) is my own function for turning a query string into a table and extracting a value. I wrote it before I discovered that (http/query-string-grammar) exists, and have not converted to it yet)

Indeed, I have just tested, with a (pp response) just before send-response, and it is not empty, as seen in the tail of the output below:

...
\x9Aw(\x8ATT!\x95\xB2\x0Fj\x8A5\a%zb\xA6\x8C\xFF\0ZI>^\x9CRhw\xB1\x18!\x81\0g\x1D\t=i\xAD\xF2\xB1=\x01\xEDNS\x82)\x8A7.O&\xA4\x9Ek\xB1B\x93\xFC#9\xE4\x8Aj)\\\x1E\x86\xA7\x97\xE5@\a\x035\x16\x01_\xC6\x9Aa\xB97\x9F\x82\xC0\x90G\x14\xA6E\x94\xB3\xBE7\x1Ep:\n\xAF\xB4\x06s\x8E\xC2\xA6\xD8\xA22@\x1443\xFF\xD9" :headers "Content-type: image/jpg" :status 200}
GET 200 /IMG?name=IMG_5435.JPG elapsed 31588.381ms
error: expected integer key for buffer in range [0, 0), got :body
  in send-response [/usr/local/lib/janet/spork/http.janet] (tailcall) on line 328, column 24
  in server-handler [/usr/local/lib/janet/spork/http.janet] (tailcall) on line 414, column 3

(note the elapsed time is huge only because it printed all the image data to the terminal)

[sogaiu]

Hmmm...

Not sure how you have everything hooked up and starting.

Specifically, I wonder about directly using send-response. If the image function is supposed to be a handler, I think it's typically supposed to return a struct or table.

I've put together a little sample with instructions on doing something similar.

Perhaps it might shed some light on what's going on?

[BillScherer]

Thanks. I'll work on it tomorrow.

[sogaiu]

The last thing that the image function does is to return the value obtained from calling send-response. The last thing that send-response seems to do is to call write-body, and the last thing that does is to return the result of calling buffer/clear.

So my guess is that the image function is returning an empty buffer because ultimately buffer/clear returns an empty buffer:

    (buffer/clear buffer)
    
    Sets the size of a buffer to 0 and empties it. The buffer retains its memory so it
    can be efficiently refilled. Returns the modified buffer.

...and thus response ends up being an empty buffer [1].

Or something like that (^^;

---

[1] Obligatory Monty Python reference about chains of inference.

[BillScherer]

Your sample was just what I needed. My code is working now, without any errors. Thank you!!!

[sogaiu]

@BillScherer A follow-up note...there were some bugs related to headers in the sample. Sorry about that.

The sample has been modified and updated to also include an example that uses router. Hopefully the bugs have been fixed (^^;