Cross compilation support #11179
Open
Cross compilation support #11179
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove QualifiyOptions Remove QualifyOptions by setting qoSetupIndependent to be always true (the current default) and qoBaseShim false (this must have been just a hack of some sort).
principalPP and setupPP seem to have gone unused since 8194fab
…ry, pkgsUseProfilingLibrary, pkgsUseProfilingLibraryShared We do not want to check the compiler.
It is not very useful.
…build dependencies
…antiatedWith from ElaboratedConfiguredPackage to ElaboratedComponent Instantiation only makes sense for components.
add stages list
…g jsem This is a user problem. User should not enable jsem on a compiler that does not support it. This change also avoid us to pass the compiler all the way down. A better approach to restore this functionality would be to defer the application of the parallel strategy.
… validateSolverResult - add step to Progress - add MonadFail Progress instance - refactor validateSolverResult
Merge fromSolverInstallPlan and fromSolverInstallPlanWithProgress.
Isolate the common logic between building and only downloading. _Push the ifs up and the loops down_
More logging in ProjectBuilding
Determine packages to build in-place by their solver id, not their package id.
Cabal uses a peculiar c program to check if LD supports and should use -x. To do this, it shells out to GHC to compiler the C file. This however requires that GHC will not bail out, yet cabal does not pass --package-db flags to this GHC invocation, and as such we can run into situations where GHC bails out, especially during GHC bootstrap phases where not all boot packages are available. We however do not need GHC to compiler a c program, and can rely on the C compiler. Fundamentally cabal does not allow modelling program dependencies in the program db, as such we must configure gcc first before using it. We make a small change to lib:Cabal (specifically the GHC module, and it's Internal companion) to allow it to configure gcc first, before trying to configure ld, and thus having gcc in scope while configuring ld. This removes the need for the awkward ghc invocation to compiler the test program.
…ectory after build
Split the function into multiple ones.
If a node has dangling edges, then the list of missing neighbours cannot be empty.
Exceptions are not nice but this is an obvious invariant. Graph should provide a better API to make this unnecessary.
Not really a fix. I do not know why this happens.
It is duplicate information since we write the program invocation right after.
|
Looks like very impressive and important work @andreabedini, it's great that you and your team have taken on this work. I'm looking forward to digging into these changes together. Would it be possible to write a cabal proposal to specify the user-facing changes to this patch? It would be good to get some feedback from other user's interested in cross-compilation and to write down the specification which this patch implements. Once a specification is established it will be easier to assess if the patch implements the agreed specification and there are suitable tests in place. From your description, it seems that someone could test this patch for themselves by using a wasm or javascript compiler and using `cabal-install" "normally"? Finally, hope you have a great holiday :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello,
this PR adds support for cross-compilation to cabal-install.
In this work cross-compilation is viewed exclusively as a cabal-install feature, and it is implemented without any substantial change to Cabal. I introduced the concept of stages (which I call build and host) with their respective toolchain (compiler and program, package and pkgconfig databases); and re-used the solver concept of scopes to separate executable dependencies from library dependencies. Executable dependencies (like setup scripts and build-tool dependencies) are solved in the build scope, while library dependencies are solved in the host scope. Roughly speaking, the build scope is for binaries you will run on the build machine while the host scope is for binaries you might want to run elsewhere (e.g. on a different platform).
As you can imagine, this involved a fair amount of work and I am well aware that this PR is too big to be considered as a whole. Furthermore I arbitrarily dropped (obscure) feature and (legacy) workarounds to simplify my work. One example being the "independent goals" solver setting: I do not think anybody is using it (or even should use it) and it was interfering with my rework of the solver scopes. That said, I do not think anything prevents bringing these things back, if only lack of time and effort.
While I am proud of how I was able to make everything fit together nicely, the PR is not without issues (
Stagedis over-engineered, build stage should be entirely optional,IsGraphis just plain awkward, etc). I welcome (and encourage!) a thorough technical review of the implementation.This work was prompted by IOG's efforts to build GHC with just cabal-install; and for this reason this branch contains a few commits that are unrelated to cross-compilation. I have spent considerable amount of time making sure each commit corresponds to a single small change1; so it should be easy to identify and separate unrelated, unneeded or unwelcome changes.
Also, shame on me, there are no tests; indeed my only testing was building GHC itself. While some tests for the solver should be easy to add, I do not have a clear testing strategy in mind. Changelog and documentation are not done either.
I have no expectations for this to be merged as-in or even with only minor changes. You are welcome to mark this as draft or to consider it a "implementation included" RFC. I will indeed be offline for the next two weeks, so go wild but I will answer any question or comment only later.
Here is an itemised list of changes, produced with the help of Claude.
Core Cross-Compilation Features
Stage System:
Toolchain Management:
Solver Enhancements:
Major Refactoring
Project Planning (ProjectPlanning.hs):
Install Plan:
Build System Integration:
Removed/Simplified Features
Legacy Removals:
Simplified Logic:
Infrastructure Improvements
Error Handling & Logging:
Code Quality:
New Configuration Options
Build Options:
Project Configuration:
File Structure Changes
New Modules:
Major Updates:
Cabal Library Enhancements
Feature Additions:
Build Improvements:
Dependency & Build Logic:
Check list:
significance: significantin the changelog file.Footnotes
and sometimes failed, see the three "all of it" commits. ↩