Internal Server Error caused by searching

[dokutoku]

Some search terms may cause server errors. For example, a search for excel-d resulted in the following message.

500 - Internal Server Error

Internal Server Error

Internal error information:
vibe.db.mongo.connection.MongoDriverException@/root/.dub/packages/vibe-d-0.9.2/vibe-d/mongodb/vibe/db/mongo/cursor.d(304): Query failed. Does the database exist?
----------------
??:? [0x555f217e44c5]
??:? [0x555f2180ab36]
??:? [0x555f217ef2ed]
/root/ldc2-1.22.0-linux-x86_64/bin/../import/std/exception.d:517 [0x555f212647e5]
/root/ldc2-1.22.0-linux-x86_64/bin/../import/std/exception.d:437 [0x555f21257315]
/root/.dub/packages/vibe-d-0.9.2/vibe-d/mongodb/vibe/db/mongo/cursor.d:304 [0x555f212b957d]
/mnt/c/Users/soenke/develop/0-dlang/dub-registry/source/app.d:462 [0x555f212ba020]
/mnt/c/Users/soenke/develop/0-dlang/dub-registry/source/app.d:322 [0x555f212b9e47]
/mnt/c/Users/soenke/develop/0-dlang/dub-registry/source/app.d:367 [0x555f212e0e06]
/root/.dub/packages/vibe-d-0.9.2/vibe-d/mongodb/vibe/db/mongo/cursor.d:233 [0x555f212b906d]
/root/.dub/packages/vibe-d-0.9.2/vibe-d/mongodb/vibe/db/mongo/cursor.d:60 [0x555f211e84dd]
/root/ldc2-1.22.0-linux-x86_64/bin/../import/std/algorithm/iteration.d:615 [0x555f212dfef8]
/root/ldc2-1.22.0-linux-x86_64/bin/../import/std/array.d:138 [0x555f211e2bf2]
/mnt/c/Users/soenke/develop/0-dlang/dub-registry/source/dubregistry/dbcontroller.d:333 [0x555f2119ac60]
/mnt/c/Users/soenke/develop/0-dlang/dub-registry/source/dubregistry/registry.d:103 [0x555f211688d3]
/mnt/c/Users/soenke/develop/0-dlang/dub-registry/source/dubregistry/web.d:476 [0x555f21168837]
/root/.dub/packages/vibe-d-0.9.2/vibe-d/web/vibe/web/web.d:1043 [0x555f21210e3a]
/root/.dub/packages/vibe-d-0.9.2/vibe-d/web/vibe/web/web.d:214 [0x555f21210a43]
router.d:218 [0x555f21651db4]
router.d:674 [0x555f216543d6]
router.d:607 [0x555f21651a86]
router.d:211 [0x555f2165183e]
server.d:2272 [0x555f21659a85]
server.d:247 [0x555f21657ebe]
server.d:239 [0x555f2165777b]
server.d:2028 [0x555f21661dce]
task.d:706 [0x555f2174d435]
task.d:724 [0x555f2174a011]
task.d:433 [0x555f217497fe]
??:? [0x555f217e55d8]

[thewilsonator]

it appears to be the presence of - in the query string

[Imperatorn]

Maybe this will be solved using the new experimental search?

I haven't tested tho

[WebFreak001]

as mentioned in #498 quoting seems to work: "excel-d"

[summer-alice]

as mentioned in #498 quoting seems to work: "excel-d"

I encountered this issue with glfw-d, adding the quotes worked as well (both double and single, the later being a non-exact search).

After a quick search, it may have something to do with how MongoDB treats hyphens as a negator (see: https://docs.mongodb.com/manual/text-search/#term-exclusion).

Searching for glfw -d, glfw -dynamic both worked as per the above documentation.

[andrewlalis]

I would like to add that the query d-properties also caused this error response.

[huglovefan]

searching for d gives the same error:

https://code.dlang.org/search?q=d (500)
https://code.dlang.org/search?q=a+b+c+e+f+g+h+i+j+k+l+m+n+o+p+q+r+s+t+u+v+w+x+y+z (works)
https://code.dlang.org/search?q=a+b+c+d+e+f+g+h+i+j+k+l+m+n+o+p+q+r+s+t+u+v+w+x+y+z (does not work)

[WebFreak001]

I think the issue there is that there are too many matches (breaks the mongodb call somehow)

[SamAxe]

Additional info, to help future people that might look into this. "It's complicated." kind of captures what I was able to find.

TL;DR -- I don't think there is a simple fix. This would be part of a bigger update/restructure to the registry code. Caveat, I spent a couple hours during BeerConf and have never used MongoDB before. If there is a simple fix, it will be by changing the query or automatically quoting the content (but that might have different error cases).

• $query has been deprecated in MongoDB, since 3.2. MongoDB is now on v6.0, possibly with changes to licenses since 3.x.
• - is a negate word symbol, but is only supposed to happen when there is a space in front of the dash. It's not clear that this isn't happening with the $query form.
• I attempted to verify that the fields were correct in database, but didn't get to bottom of why the orderby portions weren't working, even when $query wasn't used.
• Using $query with the error cases causes the ReplyFlags.queryFailure flag to be set. There is more info in the packet that results in MongoDB reply was longer than expected, skipping the rest: 230 vs. 36 message to the server.
• wireshark can capture the MongoDB traffic and decode the packets.
• $query seems like it was only required when the field contained the work "query" in it, and wouldn't otherwise be needed. Maybe it's reasonable to not use it.

The relevant code seems to be in cursor.d, in the vibe-d Mongo driver...

   if (!query["query"].isNull() || !query["$query"].isNull() )
       // TODO: emit deprecation warning
       full_query = query;
     } else {
       full_query = Bson.emptyObject;
       full_query["$query"] = query;
     }
 
     if (!m_sort.isNull()) full_query["orderby"] =
     // logError( "full_query: %s", full_query );

{"orderby":{"textScore":{"$meta":"textScore"}},"$query":{"$text":{"$search":"raylib-d"}}} is a query that breaks. This can be tested in the mongo shell by:

use vpmreg
db.packages.find( {"orderby":{"textScore":{"$meta":"textScore"}},"$query":{"$text":{"$search":"raylib-d"}}} )

However, removing the $query and the orderby seems to work.

use vpmreg
db.packages.find( {"$text":{"$search":"raylib-d"}} )

[Geod24]

I can no longer reproduce live. I tried:
excel-d
a
-d
raylib-d
And a few other variants, without success. The queries containing a dash are not really helpful but at least they don't 500.

[WebFreak001]

yeah results are not ideal, but nice to see that the issue is actually gone, although it worries me a bit that we haven't really touched the code here to fix anything