How can I get backtraces from user-supplied scripts?

[ianthehenry]

I'm working on a program that takes a user-written Janet script as input, runs it, and then does something with the result. I'd like to be able to report line:col for errors in the user script, but it seems that Janet usually doesn't provide them. For example this program:

(defn something []
  (error "example"))

(something)

error: example
  in something [playground] on line 2, column 3
  in _thunk [playground] on line 4, column 1

That gives me the line of the (something) call -- the thing I am actually interested in (imagine that something is not user-defined, but is already defined in the environment I provide to the user). But if I do anything except a top-level function invocation:

(defn something []
  (error "example"))

(do
  (something))

error: example
  in something [playground] (tailcall) on line 2, column 3

I only get the line that raises the error, not the location in the user's code.

Currently I'm just using janet_dofile, because it was easy to get started with. I'm not sure if this error tracing is a limitation of janet_dofile or of Janet itself. So far I can't come up with any workarounds that give me useful backtraces (i.e. wrapping everything in functions to try to force a stack entry (?), or fibers and manually debug/stacktraceing).

Is there a chance that wrapping everything in a function and using janet_pcall will make this better? Is there another approach I should be looking at? Or is this just not something Janet will be able to provide?

[bakpakin]

What is janet_dofile? There is no such function in the Janet C API.

[ianthehenry]

Er, sorry, I meant janet_dostring. My brain typoed that twice somehow.

[bakpakin]

The reason this happens is just tail call elimination janet_dostring. The default file runner (with dofile) does some processing on each thunk that removes the tail call.

[bakpakin]

I should also mention the janet_dostring is probably not the best way to do this, even with user provided input. If you have any significant amount of code in Janet rather than C, you should probably load your user script with Janet, using dofile or require, and then use janet_call or janet_pcall to trigger from C code (or just omit that part).

[bakpakin]

janet_dostring honestly mostly exists to help Janet bootstrap itself, and while I try to keep it fairly functional, it re-implements a lot of what run-context (written in pure Janet) does better. run-context is what require, dofile, import, etc. all use to execute code and has a lot more features like better error handling, built in debugger, etc.

[ianthehenry]

Aha, yeah, and by moving the function call out of "tail position" of the do, it preserves the frame:

(defn something []
  (error "hi"))

(do
  (something)
  1)

error: hi
  in something [playground] on line 5, column 3
  in _thunk [playground] on line 8, column 3

Is there a decent to work around this? Editing do_apply to never emit tail calls seems to fix it, and for this particular application globally losing TCE might be worth it for better error messages. But if there's a less intrusive way to do this that would be nice.

Oh, just saw your new comment. I can tryyyyy switching switching over to dofile, or more likely try adding a dostring function to Janet that I can use. I went with janet_dostring because I don't actually have the user input as a file (I'm doing something like https://janetdocs.com/playground), but I'm sure I can hack around that. I'm also not really sure how to call dofile from my C program without, like, using janet_dostring to call dofile? I guess pull it out of the (janet_core_env) table? I'll try that.

Better error handling definitely sounds worthwhile!

[ianthehenry]

There is a less serious issue where certain macros lose line information -- i.e.

(defn something [&]
  (error "okay"))

(-> 1
  (identity)
  (identity)
  (identity)
  (something))

Gives an error at the start of start of the -> form, rather than the actual line with something on it. I just discovered that issue and haven't tried to hack around it yet, though. Maybe I can convince that particular macro to preserve the sourcemap info? Or maybe this is a losing battle. But this feels like a pretty separate question I guess.