benchmark titanium against jsonld-java

[VladimirAlexiev]

Hi @filip26 !
Thanks for your implementation!
There's a good chance it'll replace jsonld-java in rdf4j, one of the most important java rdf frameworks: eclipse-rdf4j/rdf4j#3654.

However, @umbreak did some benchmarking 11m ago showing your implementation to be 10x slower than jsonld-java: https://github.com/umbreak/jsonld-benchmarks.

He guessed in jsonld-java/jsonld-java#284 (comment) that this may be due to 1.1 features, but @gkellogg countered there should be almost no slowdown for 1.0 files.

Would it be possible for you to repeat the benchmark, and if confirmed, to think about some ways of speeding up your implementation? Ontotext may be able to contribute to these tasks, depending on developments in client projects.

Thanks in advance!

[filip26]

Hi @VladimirAlexiev
the benchmark shows that 1.1 implementation/algorithm is slower than 1.0 implementation as there is no other JSON-LD 1.1 processor in Java, as far as I know.

All PRs are welcome, especially those improving performance.

BTW. there is another open issue #184 on performance.

[umbreak]

Hi. I was the one doing the benchmark. As I mentioned in the README, I just took a subset of the benchmarks from W3C that did work for both JSON-LD java (which only supports partially 1.1) and Titanium.

I can try running them again once I have a little bit of free time. I think just bumping dependencies should be enough.

[VladimirAlexiev]

@umbreak That's awful nice of you!

• How to optimize for performance #184 (comment): I'll run a simplest comparison against jena (the riot command-line tool). That's a relatively big file (10Mb) that takes 20 resp 50 minutes, and I think that NDJSON (chunking/streaming) would help there.
• But your W3C tests are small files, right?

[umbreak]

There has been a good progress on the Titanium side.
Right now the difference on the tests I've run between Json-LD Java and Titanium is ~ 5x in favour still of Json-LD Java (compared to ~ 10x from 2020). Find the updated results in my repo

But your W3C tests are small files, right?

I did not mingle with the tests provided by W3C, but just used them. I do believe that there are some larger files (depending what your metric for large is). You can download and unpack the tests used if you want to have more details. However notice that not all the files are used, because Json-LD Java did not support them all.

[filip26]

@umbreak thank you for the benchmark!

I'm closing this issue as there is an older one (#184). Please, let's move there.