Warning when conditionally loading "glue" module

[DilumAluthge]

I can conditionally load code like this:

ExampleOne.jl:

module ExampleOne

using Requires

function __init__()
    @require JSON="682c06a0-de6a-54ab-a142-c8b1cf79cde6" begin
        function hello()
            println("Hello world!")
        end
    end
end

end

But code loaded like this is not precompiled.

If I want to be able to precompile my conditionally loaded code, I can do so like this:

ExampleTwo.jl:

module ExampleTwo

using Requires

function __init__()
    pushfirst!(Base.LOAD_PATH, @__DIR__)
    @require JSON="682c06a0-de6a-54ab-a142-c8b1cf79cde6" using GlueModule
end

end

Where the contents of GlueModule.jl are:

module GlueModule

function hello()
    println("Hello world!")
end

end

With this approach, GlueModule is precompiled. Unfortunately, I get this warning:

┌ Warning: Package ExampleTwo does not have GlueModule in its dependencies:
│ - If you have ExampleTwo checked out for development and have
│   added GlueModule as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with ExampleTwo
└ Loading GlueModule into ExampleTwo from project dependency, future warnings for ExampleTwo are suppressed.

I understand the purpose of the "Warning: Package Foo does not have Bar in its dependencies" warning when Bar is a real package with a UUID that is in the Julia General registry and can be added to a Project.toml file. But in this case, GlueModule isn't a real package - it's a glue module that is located in the same package repository as ExampleTwo. GlueModule doesn't have its own GitHub repository, doesn't have its own UUID, is not separately registered in the Julia General registry, and cannot be added to a Project.toml file.

So is there a way to suppress the "Warning: Package ExampleTwo does not have GlueModule in its dependencies" warning for glue modules that are conditionally loaded by Requires.jl?

[MikeInnes]

This might be best asked on Pkg.jl or similar. I'm not sure but you might be able to give GlueModule an arbitrary UUID and add it to the "extras" section of Project.toml, or similar; so that it is explicitly there, but pkg doesn't try to install it automatically.

[DilumAluthge]

Cross posted to JuliaLang/Pkg.jl#1238

[DilumAluthge]

I'm not sure but you might be able to give GlueModule an arbitrary UUID and add it to the "extras" section of Project.toml, or similar; so that it is explicitly there, but pkg doesn't try to install it automatically.

I tried that, but unfortunately I still get the same warning.

[DilumAluthge]

Also cross posted to JuliaLang/julia#32413

[tkf]

Hi, I just came across this in Julia's issue tracker. I think I found a very evil solution to this:

module MyPlayground

using Pkg
using Requires

const gluepkg = Base.PkgId(Base.UUID("197e495a-9878-11e9-311c-51fb68a00c9c"),
                           "GlueModule")

function __init__()
    @require JSON="682c06a0-de6a-54ab-a142-c8b1cf79cde6" begin
        if Base.locate_package(gluepkg) === nothing
            Pkg.develop(PackageSpec(
                path=joinpath(@__DIR__, "..", "GlueModule")
            ))
        end
        const GlueModule = Base.require(gluepkg)
    end
end

end # module

See the full code here: tkf/MyPlayground.jl@5343249

[DilumAluthge]

Hmmm, the problem with calling Pkg.develop is that it will add GlueModule to the [deps] section of the Project.toml file of the active project. It doesn't make sense to add GlueModule to the [deps] section because it is not a dependency and is only loaded conditionally.

[tkf]

Maybe using Base.require with pushfirst!(Base.LOAD_PATH, @__DIR__) works?

[DilumAluthge]

Maybe using Base.require with pushfirst!(Base.LOAD_PATH, @DIR) works?

It looks like the signature of Base.require is require(into::Module, module::Symbol). Presumably the second argument module::Symbol should be GlueModule. What should I pick for the first argument into::Module? Should I use into = Main, into = ExampleTwo, or some other value for into?

[tkf]

What I had in mind was something like this but it didn't work (same warning message as yours). Looking at require(into::Module, module::Symbol), it has the code to emit the warning so I don't think it works either. At this point I'd stop trying to fool the manifest system if I were you.

If you are interested in precompiling conditional dependency support, see this hack by Roger-luo or an alternative idea I posted.

[DilumAluthge]

Yeah it seems like tricking Julia and Pkg is a bad idea. I think ultimately we need first-class support for this in the Julia language and/or Pkg.

[tkf]

OK I think I figured it out. As usual in programming, one more indirection helps. This seems to work:

using Requires

const gluepkg = Base.PkgId(Base.UUID("197e495a-9878-11e9-311c-51fb68a00c9c"),
                           "GlueModule")

function __init__()
    @require JSON="682c06a0-de6a-54ab-a142-c8b1cf79cde6" begin
        const GlueModule = let
            glueloader = joinpath(dirname(@__DIR__), "GlueModule", "loader", "Project.toml")
            origpath = copy(Base.LOAD_PATH)
            pushfirst!(Base.LOAD_PATH, glueloader)
            try
                Base.require(gluepkg)
            finally
                append!(empty!(Base.LOAD_PATH), origpath)
            end
        end
    end
end

The idea is to have a "loader" environment GlueModule/loader/{Project,Manifest}.toml which devs GlueModule and the main module using relative path. I then temporary put the loader environment path in the Base.LOAD_PATH while loading GlueModule.

See:
tkf/MyPlayground.jl@11d7b72

[danilo-bc]

I'm facing a similar issue and haven't found a conclusion in this thread. After issuing @require, can I do using <module_name> and then include the appropriate source files which use that module?

e.g.:

@require Makie = "ee78f7c6-11fb-53f2-987a-cfe4a2b5a57a" begin
        @eval using Makie
        include("./my-extra-plots.jl")
        export my_extra_plot
    end

I get the same Warning/Error as the original poster.
I know the problem can be solved by manually adding Makie. to every respective function inside my-extra-plots.jl, but is there a way to benefit from the fewer characters typed by using Makie?