Idea for removing / from reserved

[zkat]

I... I really am not sure this is a good idea, but I thought I'd put it here just for funsies:

We could remove / from reserved characters if we took advantage of the fact that # is reserved now, and is treated as a dispatch character:

(EDIT: single-line comments would be #!, as suggested below)

Single line comments would be #!, multiline would be #* *#, and slashdash would become hashdash #-.

With that in place, the following would become legal:

#*
this is an fstab
*#
/dev/sda1 /
/dev/sda2 /home #! home sweet home
#- /dev/sda3 /media

but:

#this is a syntax error
#"oops", you messed up, this line is also a syntax error
#true #! bool, not comment

Note that this only addresses /, so Windows paths are still screwed, but Windows supports correct slashes these days so it's probably fine.

And don't get me started about how this ship has absolutely sailed. It's probably not a ton of work to update parsers with this change but gosh it feels like some annoying logistics to suddenly push out a v3. And we should probably have a better commitment to long term stability anyway.

Previously, with credit to jax: #409

[bew]

I like this 👀

Not so sure about the #* *# for multilines, but the 2 others are great 👍

For best multiline comment, I personally love Lua's --[[ foo ]] which can support arbitrary nesting or escapes like --[=[ foo [[ no problem ]=]

So this could be #[[ multiline comment ]] 🤔

Or get inspired by markdown's backquote repetitions..
So #[ multiline ], and if you need brackets, add more around: #[[[ here [ brackets [[ no problem ]]]

[zkat]

it would still need to be #[]# because, e.g. KPath uses [] for meaningful syntax and it would be good to make sure they share a comment syntax.

[zkat]

note that /* */ already allows arbitrary nesting!

[ellieayla]

Does the above imply a shebang on line 1 would be a syntax error too?

#!/usr/bin/env kdl

[bew]

If we start re-purposing #, it might open the way for #! ... markups, and allow #!/... as valid 🤔

[tabatkins]

We'd probably just define that #! is an alternate single-line comment syntax.

[zkat]

lol that would actually be hilarious.

[tabatkins]

I agree that this is quite a change to justify a v3 so quickly, but also... I like it? Relying on # as the general meta-dispatch character for everything is pretty clean, and it means our single-line comments match Python rather than JS, which is a perfectly acceptable lateral move. (It also matches YAML and TOML.)

As I say in another comment here, I think we'd accompany this move with a declaration of #! as another line-comment syntax, to allow shebangs, which is nice.

[zkat]

I think if we coordinate things right, we can shift everyone over to "KDL RFC" mode, and declare the KDL RFC as a distinct language from KDL v2, and deprecate v2 since it has relatively little reach right now?

I feel really uncomfortable making such a huge change at all, but it... seems genuinely really valuable?

[jaxter184]

On one hand, I agree that the ship has sailed. On the other hand, this is a really nice solution that I like more than any of the solutions I proposed, and I think if enough of the users/implementers think this is worth it, it could be worth the friction to make KDL more ergonomic? For one, most URLs would be valid bare identifiers.

[tabatkins]

Yeah, impl-wise it's a trivial change, so we could do the cutover real quickly.

[zkat]

I've also created a team so we can very quickly ping all implementers about stuff like this.

[zkat]

I posted #496 about how to handle this change and other similar moments of brilliance in the future

[zkat]

Another thought I had about this. Consider these two examples:

https://example.com#!anchor
random-string#!oops I forgot a space before #!

So our choices seem to be:

1. Make #! the line comment character in which case people who paste URLs into KDL risk having their anchors silently stripped

2. Allow NON-INITIAL # in unquoted strings, but then if someone neglects to put a space after an unquoted string, the entire comment will be slurped into the string value

3. Specifically disallow #! in unquoted strings, which means pasting that link will be "safe" (syntax error, need to quote), and you can't inadvertently forget to put a space before the #! and have it get slurped in.

My main concern with 3 is that it's a very subtle nag that requires a bit more detailed knowledge of the language. It would work in like 99.9% of cases. I'm just worried about how materially frustrating and disruptive the other cases will be, and whether this means users will make a habit of scanning their URLs for #!. This can be hugely alleviated by syntax highlighting and diagnostics, of course, but I think anyone should be able to open notepad and do the right thing.

Thoughts?

[zkat]

Yeah that's definitely a footgun now but also I kinda don't feel like that's as likely as "I copied a random URL from my browser".

[zkat]

Ugh now I'm brain wormed about how many people are going to suffer because of foo=bar and I'm trying not to regret unquoted strings. 🫣

I hope it never happens in practice

[tabatkins]

Unquoted strings are a huge win, don't worry. Unquoted attributes are great in HTML too, and they can have similar issues; you just learn to reflexively quote anything "weird", it's fine.

[zkat]

@tabatkins ohhhhh actually that's right. I did remember unquoted attributes in HTML but I thought KDL would be different because it has both properties and arguments... but HTML has name-only boolean attributes, too, which effectively have the same issues.

[tabatkins]

Well, HTML boolean attrs are just the attr name, not the value, so by convention they're just simple ascii alphanums anyway and not troublesome. But I definitely put "simple" urls into attributes unquoted, but immediately reach for the quote characters as soon as there's params or anchors or anything else weird.

[bgotink]

I'm… not convinced this is a good idea. We now have four significant characters in the syntax: {} to group stuff, # is used in values and / is used in comments. (oh and \ exists as well, doing it's usual thing)
Having different characters for comments and values is imo a good thing. Using # for both values and comments makes things imo a whole lot more confusing.

As an example, we explicitly require space between all values, but not between values and comments.
So this would be a valid document:

node #"value"##* comment *##"value too"#

That's a few steps back from what we currently have

node #"value"#/* comment */#"value too"#

Another example, I'm working on a document

node "value" \
  #"raw\ value"#

and I want to comment "value" but I make a mistake and forget the -

node #"value" \
  #"raw\ value"#

is a valid document. The same can't be said for me forgetting the - in /-.

If I wrote

node #"value"
  #"raw\ value"#

did I forget to add a ! after the first #? Did I forget to escape a newline? Did I forget to use """ for a multiline string? Did I forget a # after #"value"?
Error messages will have to become more unhelpful if we implement this proposal.

---

This doesn't feel like an improvement to me. Yes, it buys us the / character for identifiers so I can write many—but not all—URLs and UNIX-style paths without quotes. I'm not sure this new syntax is a price I'd be willing to pay.

[zkat]

yeah I'm... I think I'm pretty convinced this is not a good idea, at this point. It was worth discussing, but woof, some of those cases.

[jaxter184]

Hmm yeah, these examples are pretty rough. Hopefully we can consider other options in the future for a V3 or RFC edition (maybe considering options other than # for the boolean/raw string character?), but this and the #!anchor examples have convinced me that the "simple and quick" fix seems to be more fraught than I initially thought, and have brought my hopes back down to manageable levels.

[jhasse]

Hi,

new here, just found this discussion :)

It seems that you've come to the conclusion that this is not a good idea? What about the original idea to at least allow single / in identifiers? That would still make most paths possible.

[zkat]

I've thought about that, yeah. I'm worried that it would set an expectation that could lead to some very unfortunate footguns? Like, the thing about comments is that as soon as you accidentally pop a // somewhere in there, whether through a typo or by pasting, say, a URL in, the entire reset of the node line is commented out, silently, leaving you with a partial slice of the internal data:

node /foo//bar/baz oops it's just /foo now

[jhasse]

When pasting an URL isn't that no different now, right?

node https://kdl.dev

I would say there's always a risk with unquoted identifiers. But I think many people will edit .kdl files with syntax hightlighting anyway. Also for this to go unnoticed there has to be no required arguments after the path. And the shortened path has to be "valid", too.

Implementations could also warn if there's no space before //.

[zkat]

Hmm. I mean yes, but we don't set an expectation that you can paste any kind of URL into a KDL doc right now. There's also the matter of /* and /- also being able comment syntaxes.

This all makes me really wish we'd used a different character for comments tbh.

[tabatkins]

Hindsight is always 20/20, yeah.

I agree that trying to make it possible to use a URL or path as an unquoted string is not worth the difficulty here. They're already long sequences, so the extra weight of using quotes is relatively small, and the set of non-alpha ASCII that KDL supports in idents is already "weird" enough (relative to most other languages) that I don't generally depend on it, and just defensively quote anything that contains a weird character.

[jyn514]

i don't have skin in this game, but as a survey of what other languages do - clojure uses ; as a line comment syntax and #_ as a "structural comment". that latter thing means that any "form" (e.g. (list (with (nesting))) [vector] "string" ident) can be commented out easily even if it spans many lines.

maybe that fits into kdl? you could have e.g. ;; as a single-line comment (equivalent to // today) , ;* as an inline comment (equivalent to /* today) and ;_ as a structural comment (equivalent to #_ in clojure).

[zkat]

; is tricky because it's used as a node terminator, too: node { foo; bar; baz }

[jyn514]

oh, neat. that makes things trickier yeah, even if it's grammatically unambiguous it becomes harder to write good error messages and it also looks a little odd.

[jyn514]

the only other character i see you have free is . which i think is no simpler than ; here

[zkat]

. is a valid character in identifiers lol

[simonask]

Having had to deal with cross-platform path handling a lot recently, I would personally err on the side of not bothering.

What I love about KDL is how human-parseable it is. File system paths on all platforms (but especially Windows) are so full of weird things that trip humans up, it's crazy.

1. From a usability perspective, I would worry that built-in syntax could be understood to carry the implication of some kind of validation, but that should be way out of scope (and is Very Hard).

2. In the case of serializing data to KDL, the decision on which style of string to use might become more complex. Maybe you would want to only choose unquoted-with-slashes for items that are also semantically paths (like std::path::PathBuf in Rust), but not any other string that happens to contain slashes.

3. We as programmers underestimate how many file system paths in the wild actually contain whitespace. It might be fine for KDL to be a bit programmer-brained, but it's a nice candidate for people like artists working with an asset pipeline, etc.

To be clear, I'd be fine either way, but I do think KDL 2 is particularly elegant in its simplicity.

[diondokter]

Yeah, I totally second that third point (pun intended).
On windows, pretty much all programs are stored in C:\Program Files.

[jhasse]

the think is though, that you're used to escaping spaces in paths. And you especting it for "semantics that looks like you're invoking a bunch of CLI commands!".

[sophiajt]

A couple thoughts, but apologies I'm a language person coming out of left field a bit having only dabbled with kdl. I helped create Nushell, so I've worked in systems that had to spend a lot of energy supporting bare word strings.

We tried a good handful of things, but if you're half-way to what a shell already thinks (that # is reserved and that bare words can be strings), I think you may just want to follow their rules. That would also help programmers familiar with those systems to move over. If it fits syntactically, it would be something like "a word that begins with a # begins a comment that lasts until the end of the line".

Multiline comments are something you could consider, but honestly in practice most editors can turn comments on and off if the whole lines are comments, making the older reason for including them in the language a bit obsolete these days (unless you have aesthetic reasons).

[sophiajt]

I think you can minimise footguns with the syntax highlighter helping to make clear when commenting is happening?

I think it might also be one of those things that feels odd for the first five minutes and then natural after that.

[zkat]

• Idea for removing / from reserved #494 (comment)
• Idea for removing / from reserved #494 (comment)

These two were the big concerning cases about it that made good points about its viability btw

[sophiajt]

Yeah. It seems those break down into a couple things:

• unintuitive rules for when the commenting kicks in
• surprise at hash having an additional functionality on its own in addition to being an initial and a non-initial character

For the first one, is using a lone # for comments, does it feel strange? For shell programmers, a little but likely doable. But for other folks?

I think that's where we get into the "I think it would feel funny at first, and then feel natural" hunch. It'd have to be tried out, but I think there are fewer footguns there, as it is a standalone syntax. Barewords would function with the flexibilty you want, and you already know if you put a space in a bareword, you are creating a new word. All that to say, I think the intuition would kick in and help people spot them.

I think for the second one, I think it comes down to context. As one example, the ! character gets multiple uses in Rust, as one example. By itself it's bottom, on a unary operator it's not, and prepending a trait it means absence of.

Are there cases, like the second link showed, where things get a bit weird?

# #false #true #"foo"#

Yeah that looks strange. But how often will that come up in practice?

If in practice most of the time comments are documentation on configuration lines, I think it will flow:

# set up the Linux box
foo #true #"Trans Girls Love Linux"# bar ./baz

Again, especially if syntax highlighting helps the eye adjust.

[zkat]

I think I'm concerned about stuff like this:

foo \
  #false by default
  my-flag

This results in the following KDL doc, with this scheme, translated to JSON:

[
  { "name": "foo", "args": [false, "by", "default", "my-flag"], "props": {}, "children": []}
]

I'm genuinely worried about the requirement of having that space for dispatch, although it's technically a dispatch character. It might be a matter of me just worrying too much and it'll feel fine in practice?...

[sophiajt]

It'll likely be a tradeoff. The chance of a footgun catching someone vs the utility of the syntax that doesn't dip into your other constraints.

In this case, I think folks will naturally start putting a space after # but you're right that it's possible someone might accidentally both not put the space and use a keyword at the same time. It could happen. I think it's probably a rare case, but it could happen.

[jaxter184]

I've since found a workaround for my Open Sound Control use case (replace all .s with / after parsing unquoted strings) so I think I'm perfectly OK with this not being a feature, but since we're revisiting the topic, here's another thought I had long after the initial discussion:

Maybe have some way to replace special characters at a per-document level? For example, if I wanted to use \ or | or ! as the "value" character, adding a header to the document as the first non-shebang line (preferably something that errors out in KDLv1 in case someone tries to use it with something that doesn't support it) could tell the parser that this is using a different set of characters.

#!/usr/bin/env shebang
=== value=\ comment=#
node/name/with/slashes \true \\"raw string syntax is maybe ambiguous this case"\\ ##this is a comment

or

=== value=| comment=?
node/name/with/slashes |nan ||"less ambiguous here, though maybe less "familiar"..."||
?-all comment types would work the same, just with the "main" character swapped out

Downsides:

• Burdens parsers and syntax highlighters with extra features to implement
  • There would have to be some sort of logic to make sure that users don't create an ambiguous syntax
  • That being said, due to the header, it is largely backwards compatible, in that old documents work with new parsers, and new documents error out old parsers instead of silently failing. Even if this feature isn't implemented, I think having this sort of header might be useful for metadata if KDL ever needs that in the future.
• Throws all recognizability out the window; it would be harder to tell out of context what is and isn't a KDL document

Upsides:

• Might make KDL more concise in certain situations