Consider JRE provided JAXP implementation over deprecated Xerces for XML parsing

[drkstr101]

Hello, I have been tasked with eliminating CVE-2012-0881 and CVE-2013-4002 from our automated vulnerability report, which I have traced down to a Xerces dependency brought in from this project.

While Xerces was the defacto XML parser for quite a long time, it has been considered somewhat deprecated (IMHO) since Java 1.8, and it is now suggested to use the built-in implementation of JAXP provided by the JRE. I've taken a quick peek at the code and see that Xerces is referenced directly, so replacing it to use standard APIs may take a bit of work, which I would be willing to assist.

If replacing Xerces is impractical, please consider pinning xerces:xercesImpl:2.12.2 or later to address open vulnerabilities. Upgrading to the newer JAXP API would still be advisable in this case, for future-proofing reasons.

Please also understand that I'm not an expert on this topic, so I could be mistaken about any or all of these claims.

Sources

• https://stackoverflow.com/questions/12480046/whats-the-difference-of-jaxp-jdom-dom4j-and-xerces
• https://stackoverflow.com/questions/68703602/maven-project-using-xerces-2-with-java-11
• https://issues.apache.org/jira/browse/XERCESJ-1706
• https://stackoverflow.com/questions/11677572/dealing-with-xerces-hell-in-java-maven
• https://xmdocumentation.bloomreach.com/library/upgrade-minor-versions/remove-apache-xerces.html

[flavorjones]

@drkstr101 Thank you for asking these questions.

CVE-2012-0881 has been resolved since Nokogiri v1.9.0: #1831

It may be possible to upgrade xerces; I'm really looking for help maintaining the JRuby implementation since I'm not a JRuby user myself. Are you interested in helping out?

[drkstr101]

Yes, you are correct. It was a mistake for me to bring the version bump, or really any specific CVE at all. It destracts from my main point, which is that there has been no reason to explicitly include Xerces and reference the old and outdated namespaces in code since at least Java 1.8.

To my understanding, Xerces is now part of the official JRE under a new namespace, and that implementation will always be more up-to-date and secure than the unofficial maven package being pulled in here. Our preference would be to exclude this dependency entirely. This works for the few other dependencies we have that are pulling in this package since they are using the new namespaces, and if anything, use a Java property to assign the underlying sax parser to one in the internal Xerces namespace while still using the new API in code. This allows them to continue working when a rule is added to exclude that package. Nokogiri is one of the few (if not only) holdouts in our dependency chain still explicitly referencing the old namespace.

We can all get off this crazy train once we all agree to stop using this unofficial maven package that packages old Xerces in a way that's no longer needed. It was more of a humble request than any demand for action, which is why I brought it up for discussion.

[flavorjones]

I need help with the Java implementation. If you'd like to submit a PR I would welcome it very enthusiastically!

[drkstr101]

I was going to take a stab at it today. There may be good reason to stick with Xerces here, supporting old versions of Java for example, or depending on some obscure implementation detail that is slightly different in the new API. I did see a few reports of this nature during my research. I don't see anything super fancy here though, so I'm pretty sure it will be an easy migration. I will report back with the results.

[flavorjones]

@drkstr101 Wow, thank you. If you haven't already seen it, you may want to take a look at https://nokogiri.org/CONTRIBUTING.html#bumping-java-dependencies which talks a bit about updating the java dependencies.

[drkstr101]

We almost got off easy here with a simple find/replace as I did confirm there are full implementations of both Xerces (com.sun.org.apache.xerces.internal) and Xalan (com.sun.org.apache.xalan.internal) in the java.xml module from the JRE. There would be some work needed to expose these internal classes when using Java-9 modules (JPMS), but so far, so good.

The problem I ran into was with the CyberNeko HTML Parser. This library also depends on the original org.apache.xerces namespace, and I'm not sure of yet the best way to handle this. For this migration to be practical I would think any changes to the underlying behavior are unacceptable. I will need to experiment a bit more to see if a sutible replacement even exists for HTML parsing.

I'll be sure to keep you posted. Cheers, and thank you for the encouragement! :)

[drkstr101]

I've spent a bit more time understanding the feature requirements, guiding principles, etc. of Nokogiri, and feel it would be worth the effort to produce a Java extension rewritten to use nothing but native public APIs for XML/DOM parsing. I've looked into Neko, and I'm pretty sure it's just using the underlying DOM parser in Xerces, so we should have no problems there. All said and done it should behave exactly the same as it did before the rewrite. Otherwise I would say the project is a failure.

It may take a while since I will be doing this in my spare time, but I think it would be an interesting fun little project with good learning potential. I will probably have a ton of questions and may need a bit of assistance. I hope that's OK.

Cheers!

[flavorjones]

@drkstr101 That's exciting and encouraging to hear! Please do let me know if I can be of any help.

[flavorjones]

I'll also tag @kares and @headius for their awareness. Both are JRuby commiters who've contributed their time and knowledge to improve Nokogiri over the last few years and I'm hopeful would also be willing (time permitting) to offer advice to you here.

[drkstr101]

Hello again. There's been some renewed interest in this where I work, so I wanted to let you know where I got stuck. I first tried a simple find/replace on the imports to the JDK internal namespace. This works for Java 1.8 and 11 but breaks in 17 (or versions therein in between). So, I started to look into what it would look like to do a complete migration to the JAX-B API but hit a roadblock with finding a suitable replacement for nekodtd that appears to be from the CyberNeko HTML Parser. One option for moving forward is finding a replacement dependency that satisfies the original. Another would be to fork or vendor this unofficial mirror created "from the latest tar file that is in the download section."

I'm hoping to have a bit more time to work on this and get a merge request for review by the end of the year. I also realize this is a fairly risky proposition, and you are, of course, under no implied obligation to accept these changes.

[flavorjones]

My interest in maintaining the JRuby implementation of Nokogiri is flagging, so any help you're willing to put in is very welcome and would likely be an improvement over what I've been able to do on my own.

The nekodtd mirror you've referenced was created by and for this project (Nokogiri), so you should feel free to consider it mutable if you think that using it would be the best path forward.

Please let me know if you want to talk anything through. I don't know the JAX-B API or the java libraries well, but can be a rubber duck and/or pull one of the JRuby folks in as needed.

[drkstr101]

Ha! I should have noticed the namespace there. That makes things easier. Thank you. 😊

[headius]

@flavorjones I can jump in here and try to help keep this moving forward, as well as helping with Nokogiri JRuby maintenance as we flesh this work out. We appreciate how much effort you've put toward this over the years and I want to make sure the JRuby support does not fall behind.

@drkstr101 I did not see this update when @flavorjones tagged me, but it sounds exciting and I would be happy to help you with any questions or guidance you need! Anything we can do to simplify the JRuby version of Nokogiri would be very welcome. You may want to join our Matrix chat room (https://matrix.to/#/#jruby:matrix.org) and ping me directly there.

[flavorjones]

Thanks for your help and support, @headius!

[drkstr101]

Excellent, thank you @headius! I admit my knowledge of the JRuby extension API is relatively limited, but I've done other FFI-type stuff before. Does this page seem like a reasonable place to start? Can you recommend anything more official?

I found it pretty straightforward, and I don't see it being an issue, but if you are aware of any resources not easily discoverable by Google, please share them. Cheers, and thank you!!

[headius]

That is indeed a good preview of how to make a JRuby extension. You can also look at extensions that already exist in Nokogiri.

However we may be able to do a lot with pure Ruby. JRuby can import and call Java classes from Ruby just like any other Ruby class, using a bit of extra syntax:

java_import 'java.lang.System'

System.exit

There's plenty of examples of Java integration on the JRuby wiki (wiki.jruby.org) and I can help if you have other questions. Stop by the chat if you need anything!