Tests fail on v0.5

[ranjanan]

Tests fail for me on v0.5 rc2:

Error During Test
  Test threw an exception of type MethodError
  Expression: to_dot(attrs) == "[\"foo\"=\"bar\"]"
  MethodError: no method matching similar(::Dict{String,Any}, ::Type{String})
  Closest candidates are:
    similar(::SubArray{T,N,P,I,L}, ::Type{T}, ::Tuple{Vararg{Int64,N}}) at subarray.jl:52
    similar{T}(::Array{T,1}, ::Type{T}) at array.jl:122
    similar{T}(::Array{T,2}, ::Type{T}) at array.jl:123
    ...
   in _collect(::Dict{String,Any}, ::Base.Generator{Dict{String,Any},Graphs.##33#34}, ::Base.EltypeUnknown, ::Base.HasLength) at ./array.jl:318
   in to_dot(::Dict{String,Any}) at /Users/ranjan/.julia/v0.5/Graphs/src/dot.jl:66
   in include_from_node1(::String) at ./loading.jl:426
   in macro expansion; at /Users/ranjan/.julia/v0.5/Graphs/test/runtests.jl:28 [inlined]
   in anonymous at ./<missing>:?
   in include_from_node1(::String) at ./loading.jl:426
   in process_options(::Base.JLOptions) at ./client.jl:262
   in _start() at ./client.jl:318
ERROR: LoadError: LoadError: There was an error during testing
 in record(::Base.Test.FallbackTestSet, ::Base.Test.Error) at ./test.jl:397
 in do_test(::Base.Test.Threw, ::Expr) at ./test.jl:281
 in include_from_node1(::String) at ./loading.jl:426
 in macro expansion; at /Users/ranjan/.julia/v0.5/Graphs/test/runtests.jl:28 [inlined]
 in anonymous at ./<missing>:?
 in include_from_node1(::String) at ./loading.jl:426
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /Users/ranjan/.julia/v0.5/Graphs/test/dot.jl, in expression starting on line 22
while loading /Users/ranjan/.julia/v0.5/Graphs/test/runtests.jl, in expression starting on line 25

[tkelman]

What is still using this package? It is not actively developed or maintained and should be considered effectively deprecated.

[papamarkou]

@tkelman, despite this package not being actively maintained, it is used by other packages (including Lora that I am developing), so it is a dependency for other packages, and for this reason Graphs can't be deprecated.

[dehann]

IncrementalInference.jl package also heavily uses Graphs.jl

[tkelman]

Then the question is why are you using an unmaintained package that doesn't pass its own tests when an multiple actively maintained alternatives are available?

[papamarkou]

@tkelman, if you have the time to make the necessary software changes and submit a PR in Lora that migrates functionality from Graphs to LightGraphs, you are welcome to do so. I had spent a few hundreds of hours to software-engineer the graph-based model of Lora on the basis of Graphs, which required both conceiving and implementing the Graphs-centric approach. I simply don't have enough time to move to another package for accommodating the graph models of Lora. I think this answers your questions as to why things are the way they are.

Although Graphs is not actively maintained, it has sufficient functionality to be used by the community. Besides, developers don't have the time to switch packages depending on which packages in the ecosystem fall in or out of flavour or fashion, especially when we are talking about packages like Graphs, which have required hundreds of hours to develop in the first place and ship with important functionality. In an imaginary situation, if my code in Lora had become compatible with another graph-related package (which is not possible anyhow for the reasons explained above), then what would happen if the developer of a new graphs package stops maintaining it in turn, would I have again to migrate to a third graph-related Julia package?

One possibility is that an attempt is made in the next GSoC or JSoC to recruit a student to maintain and update Graphs, considering how important and good this package is anyhow, as I think you haven't appreciated enough the quality and importance of Graphs when you suggested it is deprecated. The second alternative is that we simply ensure that Graphs, as it stands, can be used across new Julia versions (and if someone comes along to make more serious maintenance of it down the road, all the better). The third option as you proposed it, which is not really an option, would be to deprecate Graphs, but then Lora will effectively not be working anymore, and as I explained I have my own roadmap for Lora and no time to change to another graph-based package - Lora is used by the community, I use it for conducting research, writing research articles and possibly a book. If you take such a drastic measure with Graphs that renders Lora unusable, you are effectively asking me to stop using Lora or Julia for that matter, which I don't think it is a nice thing to ask.

[tkelman]

In #218 you said you would "do a bit of housekeeping, at least to ensure that the package remains alive." 2 small PR's were merged, but other than that? Given Lora seems to work on 0.5 as of its latest tag, at least some of Graphs.jl apparently works there. All this package needs is some active maintenance keeping it up to date, then I wouldn't be calling it unmaintained or effectively deprecated.

https://github.com/JuliaLang/Graphs.jl/graphs/contributors speaks for itself, it's not a question of fashion or importance or quality, it's a question of ongoing engineering hours being put into it. Deprecating the package would not change anything for users of existing versions of Julia, it would only put an upper Julia version ceiling on whether or not the package is installable.

[papamarkou]

Yes, my statement about housekeeping Graphs had been ambitious given my time constraints. Same goes for Sundials, I said about 3 years ago that I would contribute more actively to it too. If we trace my github trajectory, I am sure we will uncover more online messages of mine along these lines. Bottom line, it happens frequently to find oneself overwhelmed with work and not being able to contribute to multiple fronts. Although I am not sure I am not the only one who spreads thin across packages or turns to be too optimistic about their levels of contribution, I can apologize if it makes anyone feel better, or simply be twice as careful with my (promises for) level of involvement if it helps. At the same time, we are all aware that I am not a full-time employee of Julia, I just use it and develop with it as much as I can as part of my own academic work, so I can't be seriously held accountable about Graphs or any other important package if I am not the actual creator of it.

This is what I am trying to explain, that it is not acceptable to put an upper Julia version limit on Graphs, this will effectively prevent Graphs being easily installed in upcoming Julia versions. Please, do not do that. Let's just ensure that Graphs is tweaked as much as it is needed so that it remains simply compatible with new Julia versions. Thanks.

cc @StefanKarpinski @ViralBShah

[tkelman]

There was actually a GSoC project working on a graphs package, just not this one - https://github.com/pranavtbhat/Graft.jl. Any such project would need a mentor, and when a GSoC involves working on an existing package then that would need to be someone with the appropriate time for code review and assisting with maintenance.

You do need to be at least a little bit accountable for your dependencies when you decide to start using them, especially ones that run the risk of becoming unmaintained and no longer working.

[papamarkou]

Right, when I used Graphs as a dependency, it was in JuliaLang (as it still is) and it was developed at the time by one of the core Julia members of the community, @lindahua. I wouldn't say that at the time Graphs seemed to be at risk of becoming unmaintained given these facts, quite the opposite.

I would not have the time to consistently supervise a student for Graphs myself.

In my humble opinion, maintaining Graphs just so that it is not deprecated is at this stage the most reasonable avenue. I would like to kindly ask you not to deprecate Graphs, but I can't do or say anything else other than ask for this. Many thanks.

[ViralBShah]

Clearly, Graphs.jl is not getting maintained, and if tests are not passing, does Lora.jl want to be depending on it? I am ok to keep it under maintenance as long as it keeps receiving updates with julia versions and keeps passing its tests.

Do we know what the effort is to make it work again? If so, we can leave it as it is.

[ViralBShah]

@ranjanan and I are looking into what needs to happen for 0.5. Hopefully not much.

[papamarkou]

It depends really on what level of maintenance we have in mind. If by maintenance we basically want to ensure that Graphs.jl is not broken between versions, that is probably very minimal work and I can certainly look into it @ViralBShah. If not mistaken, what @tkelman means is that there are some more serious matters of maintenace, such as bug fixing and active package development, that's what I don't manage to find the time to do.

I can see the point of avoiding dependence of Lora on a non-actively developing package, as Graphs has ended up being, but migrating to LightGraphs is not entirely trivial time-wise... plus Graphs, despite its non-hyperactive state, I think it is a good package.

[ViralBShah]

Well, the maintenance should be what is required to keep Graphs.jl free of deprecation warnings and passing the testsuite. I don't think @tkelman means maintenance as active package development, but we certainly have to fix bugs that come up - which can be addressed as happens.

[papamarkou]

I will have a quick look now to see if deprecating warnings and test passing is easy to fix, hopefully it will be ok. Bug fixing can be a bit trickier because it requires a better reading of the package's code - I do have a ground understanding of it as I needed to develop my own graph type, but certainly I don't know deeply all the nitty-gritty details of it so as to be proficient at its bug-fixing.

I will let you know in the next few mins how easy it is to fix deprecation warnings and test failures.

[ranjanan]

@Scidom @ViralBShah I found the issue. On v0.4:
map(a -> (a[1],a[2]), dict) creates an array of tuples of all keys and values. This doesn't work on v0.5. An appropriate fix would be doing map(a -> (a[1], a[2]), [dict...]), which explicitly constructs an array of Pairs.

I'm working on a PR for this.