Launching: 'Include required Plug-ins automatically while launching' should include fragments like when pressing the 'Select Required' button

[sratz]

In the launch config dialog,

the set of bundles included when [x] Include required Plug-ins automatically while launching is checked does not include fragments:

eclipse.pde/ui/org.eclipse.pde.launching/src/org/eclipse/pde/internal/launching/launcher/BundleLauncherHelper.java

Lines 560 to 562 in 66a95fd

	DependencyManager.Options[] options = includeOptional //
	? new DependencyManager.Options[] {DependencyManager.Options.INCLUDE_OPTIONAL_DEPENDENCIES}
	: new DependencyManager.Options[] {};

But when pressing the Select Required button, fragments are included:

eclipse.pde/ui/org.eclipse.pde.ui/src/org/eclipse/pde/internal/ui/launcher/AbstractPluginBlock.java

Lines 875 to 877 in 66a95fd

	DependencyManager.Options[] options = fIncludeOptionalButton.getSelection()
	? new Options[] { Options.INCLUDE_NON_TEST_FRAGMENTS, Options.INCLUDE_OPTIONAL_DEPENDENCIES }
	: new Options[] { Options.INCLUDE_NON_TEST_FRAGMENTS };

This is confusing.

And especially, since the recent changes in #1852, that made the default 'The plugin of the Test', this has become fragile, because now needed fragments might be missing and the test may not launch at all.

My proposal would be to make [x] Include required Plug-ins automatically while launching behave like the Select Required button, and include fragments.

[sratz]

@laeubi @HannesWell what do you think? I can provide a PR if you agree.

[HannesWell]

My proposal would be to make [x] Include required Plug-ins automatically while launching behave like the Select Required button, and include fragments.

The current behavior is intentional and I strongly discourage from changing it. Fragments can have new dependencies and would then pull in new bundles. And there are cases where the set of new bundles is very large.
The strategy for fragments (just like for optional dependencies) is that one has to include them explicitly in a launch if they should be in the launch. This has been discussed at length when this was introduced via Bug 544838 and related issues (I don't find the point about fragments anymore, maybe this was discussed in Gerrit and is now lost).

But when pressing the Select Required button, fragments are included

That's just for historic reason as it was like that since ever. If you think the difference is a problem, I'm open to change that as well.

And especially, since the recent changes in #1852, that made the default 'The plugin of the Test', this has become fragile, because now needed fragments might be missing and the test may not launch at all.

Doesn't this include the test-plugin/fragment itself, which pulls in all dependencies (like a host)? If a test requires some bundle, then it should specify it as a dependency (which is admittedly difficult at the moment for fragments)

[sratz]

The concrete problem is the following. We have the following dependency chain:

com.sap.adt.foo.bar.test
- com.sap.adt.foo.bar
  - com.sap.conn.jco
  - com.sap.conn.jco.win32.x86_64 [fragment]
  - com.sap.conn.jco.linux.x86_64 [fragment]

The matching native fragment is necessary.

But in the current state, where the default is now that only the required plugins are added to the JUnit Plug-in test launch config (#1852), when we launch a test from within com.sap.adt.foo.bar.test, we end up with only

com.sap.adt.foo.bar.test
- com.sap.adt.foo.bar
  - com.sap.conn.jco

and the test fails to launch, because the native fragment is missing.

In principle this is the same as

- org.eclipse.swt
- org.eclipse.swt.win32.win32.x86_64
- org.eclipse.swt.gtk.linux.x86_64

[laeubi]

@sratz I recently enhanced the options for launch PDE and it allows to select the default options now AS well when create New launch configs. According to my testing this has worked for fragments as well. Beside that, if a Fragment is required one should usually declare a required capabilty so it can be discovered automatically as done with the ne SVG support.

[sratz]

Beside that, if a Fragment is required one should usually declare a required capabilty so it can be discovered automatically as done with the ne SVG support.

How is this done for the SWT native fragments? E.g. org.eclipse.swt.win32.win32.x86_64 does not provide any capability. Why are those included somehow?

[merks]

They do this so that the host requires the fragments:

https://github.com/eclipse-platform/eclipse.platform.swt/blob/master/bundles/org.eclipse.swt/META-INF/p2.inf

But it does seem to be that discovering fragments (with host requirements) whose host is present would be in general a good thing.

[sratz]

They do this so that the host requires the fragments:

https://github.com/eclipse-platform/eclipse.platform.swt/blob/master/bundles/org.eclipse.swt/META-INF/p2.inf

But p2.inf is not considered by PDE in a launch config for a JUnit Plug-in test, is it?

[merks]

That's a good question. I am expecting that PDE is generally using the p2 publisher to turn all the MANIFEST.MF and feature.xml along with p2.inf advice files into the metadata; but maybe that's just an assumption.

[HannesWell]

But p2.inf is not considered by PDE in a launch config for a JUnit Plug-in test, is it?

That's right.

That's a good question. I am expecting that PDE is generally using the p2 publisher to turn all the MANIFEST.MF and feature.xml along with p2.inf advice files into the metadata; but maybe that's just an assumption.

PDE just relies on the information from the feature.xml and MANIFEST.MF and doesn't use the P2 publisher at all, at least not when it comes to launching products.

and the test fails to launch, because the native fragment is missing.

Understand, this indeed does not work at the moment.
For now setting in the MANIFEST.MF of the host-bundle Eclipse-ExtensibleAPI: true should help you with:

• Include fragments when computing dependencies for Extensible-API bundles #1853

On the long run, we should better use the capability/requirement approach suggest by Christoph. I intend to propose a corresponding solution for SWT, but it will also require adjustments in Tycho to break the cycle at build time, that exists when a host-bundle requires one of its fragments (but this just as a side-note).

[sratz]

But p2.inf is not considered by PDE in a launch config for a JUnit Plug-in test, is it?

That's right.

Do you think it would make sense if PDE would consider p2.inf as well, when computing dependencies?

This would align PDE more with Tycho and would also help us in certain other scenarios where tests are not written perfectly and rely on certain other test fragments of dependencies to be available as well.

For now setting in the MANIFEST.MF of the host-bundle Eclipse-ExtensibleAPI: true should help you with:

• Include fragments when computing dependencies for Extensible-API bundles #1853

Ah thanks! That's why it works for SWT! Somehow missed this PR in my investigation. We'll add this flag as well to our connectivity bundle which should then improve the situation.

[vogella]

I see also problems in a new workspace with the Eclipse RCP template and required plug-ins #1933

[laeubi]

If anyone requires old behaviour this can be configured in the preferences see N&N

[laeubi]

So just to wrap up some things:

1. The main problem is that a fragment can not be required
2. If the test fails because of a missing fragment, then this usually means that the problem is already there and usually only works because some other techniques are used (e.g. p2.inf)
3. p2.inf can not easily be supported because PDE does not use P2 as its dependecy resolution but an OSGi resolver. Tycho works differently here and uses a combination of P2 and OSGi on different levels. Also Tycho has a Taarget Platform per project, PDE has a global one per workspace
4. If we want that fragemnts are considered always we would need a new option here (PDE already has something linke this internally that try to guess if a fragment is a test fragment or not, configurable in the preferences and by some guessing of dependencies)

In general we should check that platform itself works with this feature, so if there are any general issues (that are independent from the changes we made) we should have deicated issues for that (and that then might be adopted by others as well).

Apart from that PDE/Eclipse uses some non standard extensions like Eclipse-ExtensibleAPI: true what is actually a bit bad and we better should replace it with some standard techniques. In the meanwhile changing the default to include all workspace plugins in the prefrences might be a solution.

[merks]

Is there a possibility to check all plugins (fragments) for those that have a host requirement on a host that is otherwise included in the launch? That information is available without p2, right?

[laeubi]

Is there a possibility to check all plugins (fragments) for those that have a host requirement on a host that is otherwise included in the launch?

I'm not completely sure about the desired algorithm you have in mind, so you mean if bundle x is explicitly included by the user? I'm not sure this is really a good measure.

Also one should mention that of course it is possible to go to the launch config and include any fragment manually there, it just don't work automatic.

Given that I think fragments are most often used in the context of native code fragments, I could think about that PDE not only looks at Eclipse-ExtensibleAPI: true (what is defined by the host) but at Eclipse-PlatformFilter (what is defined by the fragment), so a good aproach to me seems:

1. If there is a fragment F for Host H
2. It has a Eclipse-PlatformFilter that matches the current launch environment
3. H is included in the transitive dependencies of the launch
4. then F should automatically be included