BUGFIX: Skip content with hidden parent nodes in fulltext index

[Sebobo]

With this change nodes with hidden parents of type content are skipped during indexing.
By checking the visible state, the hidden flag and the time based visibility are checked.
Document nodes, which are full text roots are already checked, so we don't need to traverse the tree to the site node.
The performance impact should be negligible as the parent nodes are usually already in the cache.

As the fulltext indexing is a snapshot to make the time based visibility fully work. There needs to be a scheduled reindexing.

Relates: #214 #377

[dlubitz]

This would change the current behavior, as currently also hidden nodes get indexed - and filtererd out on quering.

With this change we would have a mix of nodes which might be hidden itself, but no nodes with hidden parents. Also I'm not sure how this would work with re-indexing on change of a given node. You would need to update all child nodes too, to get this working without recreating this index.

From my POV there is no easy way to fix this issue, because there is a problem in the concept.

[Sebobo]

Fulltext indexing always includes all childnodes. To prevent side effects from this change we could only apply the filter to nodes that are not marked as root in their search config.

[dlubitz]

Fulltext indexing always includes all childnodes

Content nodes yes, but not child document nodes.

[Sebobo]

I don't see the problem. The full text root nodes can be checked at runtime which is already happening I think. This covers their subparts.

[so-grimm]

I know you're still discussing whether this is even a viable solution or not, but since it is for my use case, I tested it in my project. It didn't work for the way my nodes are set up, because there's often a ContentCollection between the childNode that's not supposed to be indexed and the contentNode that is hidden. So I think you should probably allow both Neos.Neos:Content and Neos.Neos:ContentCollection in your while condition. Without it, it immediately exits or rather doesn't even enter the loop (at least in my setup).

[Sebobo]

I know you're still discussing whether this is even a viable solution or not, but since it is for my use case, I tested it in my project. It didn't work for the way my nodes are set up, because there's often a ContentCollection between the childNode that's not supposed to be indexed and the contentNode that is hidden. So I think you should probably allow both Neos.Neos:Content and Neos.Neos:ContentCollection in your while condition. Without it, it immediately exits or rather doesn't even enter the loop (at least in my setup).

You are correct. And as I said before I think only the check for the full text root should be done independent of any nodetype. I'll adjust the PR.

[Sebobo]

I updated and retested the code. It now checks for the full text roots, so we are independent of the nodetypes.

But I found another issue related to this. When making a parent visible again, its child nodes are not added to the index again if the parent is not a full text root. This should be fixed in another PR imo.
As soon as the child nodes are edited again, their content reappears in the search.