Serialization recommendation for booleans?

[gpdf]

I find the stereotyped paragraphs about VOTable serialization at the end of almost every subsection of Section 3 very useful. For example:

Floating point interval values serialised in VOTable or service parameters must have the following metadata in the FIELD element: datatype="double" or datatype="float", arraysize="2", xtype="interval".

I recommend adding the same kind of statement in Section 3.2, Boolean.

[pdowler]

The section on Numbers right before it simply refers to VOTable and I don't recall why this section doesn't also do the same. I think it is related to the fact that around ADQL-2.0 / TAP-1.0 we did not have full boolean support in ADQL so we used integer values 0 and 1. You see it in the ADQL contains and intersects functions and in the principal, 'indexed' and 'std` columns in the tap_schema.columns table. #ugh

Ideally, this section should just refer to VOTable rather than XSD.

[pdowler]

I could imagine including something like this to grand-father in that previous usage:

datatype="boolean" for pure VOTable usage (true|false?)

datatype="int" xtype="boolean" for legacy ADQL/TAP usage (0|1)

but I don't know if it is worth it.

[msdemlei]

On Mon, Apr 28, 2025 at 01:06:13PM -0700, Patrick Dowler wrote: pdowler left a comment (ivoa-std/DALI#41) I could imagine including something like this to grand-father in that previous usage: `datatype="boolean"` for pure VOTable usage (true|false?) `datatype="int" xtype="boolean"` for legacy ADQL/TAP usage (0|1) but I don't know if it is worth it.

Let's not do it unless we have a clear idea what we fix by that. Also note that ADQL's boolean support still sucks (you can't write "where bool_colref", for instance, and it'd be a nightmare to fix it). On the other hand (and given that fact), you'd have my vote for deprecating the boolean datatype (and in particular its arrays of them) in VOTable and *then* introduce the boolean xtype. But even there I'm not sure it's worth the effort. On the long run, I think we'd have a bit less wonkyness in implementations (I'm sure DaCHS's booleans will show some interoperability problems when VOTables with booleans come around, which fortunately by and large they don't...)

[gmantele]

I don't like the introduction of xtype="boolean". DALI should not try fix ADQL by having a XType boolean. As you say @pdowler , VOTable provides the datatype boolean, so we don't need the xtype boolean. It is too confusing.

I'd say, if boolean should be fixed somewhere, it should be in ADQL and TAP. As said @msdemlei , boolean constraints are not easy to have in ADQL and that's why this feature has been frozen until now. Let's see in a future version if we can do it anyway.

About TAP, we could decide to have columns like principal returning a boolean instead of an integer, but that would be an issue for some databases which do not provide this datatype (e.g. sqlite). But one may say, that TAP services based on such database can do some trick to return VOTable's booleans when integers known as boolean are encountered. Not trivial, but possible. Anyway, I'd suggest to keep TAP as it is now. I am not sure it's worth it.

[mbtaylor]

Let's not do away with VOTable datatype="boolean". It's been there a long time and it's not doing any manifest harm, and it's in common usage, for instance there are 12 boolean columns in gaiadr3.gaia_source at ESAC.

[gmantele]

I forgot to say it as well, but like @mbtaylor , I am not in favor of dropping datatype="boolean". It is there for a so long time. It would probably be a major change for clients and servers if we drop it.

[pdowler]

I completely agree that datatype="boolean" aka normal VOTable usage should be encouraged.

The other form is more like what we actually did in the past (and still do) in ADQL and TAP, so it could be a way (probably not a good one) to improve the current situation:

<RESOURCE type="results">
    <INFO name="QUERY_STATUS" value="OK" />
    <INFO name="QUERY_TIMESTAMP" value="2025-04-29T16:11:00.097" />
    <INFO name="QUERY" value="select top 1 std,principal,indexed from tap_schema.columns" />
    <TABLE>
      <FIELD name="std" datatype="int">
        <DESCRIPTION>a standard column; 1 means true, 0 means false</DESCRIPTION>
      </FIELD>
      <FIELD name="principal" datatype="int">
        <DESCRIPTION>a principal column; 1 means true, 0 means false</DESCRIPTION>
      </FIELD>
      <FIELD name="indexed" datatype="int">
        <DESCRIPTION>an indexed column; 1 means true, 0 means false</DESCRIPTION>
      </FIELD>
...

side note: descriptions above improved slightly from what our TAP services actually say, which is kind of embarrassing :-)

The main issue here is that the current boolean section refers to XSD instead of VOTable, which has somewhat complicated serialization of boolean (different in TABLEDATA and BINARY). I will fix and flag @gpdf to review the PR.

out of scope: Fixing the use of boolean in TAP (those tap_schema columns) and ADQL (those functions) is a separate issue since. I will create an issue in TAP repo for the type of those columns in the tap_schema.

[pdowler]

This was fixed in the minimal way in PR #51