Upgrade Lucene

[mcserep]

We use Lucene 4.9.0 in the search parser, while the current major version is 10.

The goal of this ticket is to analyze that what breaking API changes and dependency changes would be required to upgrade Lucene to a newer version.

[LakatosMark1]

Lucene 5.5.X: Csak az importok és a verzió-konstansok cseréjét igényli, nem kell újraírni az adatbázis sémáját, és a régi indexek is megmaradhatnak
Lucene 6.6.X: Minimum Java 8, a régi számkezelés (NumericField, IntField) végleg megszűnik, az új PointValues és DocValues logikát kell használni
Lucene 10.X: Java 21: újra kell írni a teljes indexelőt és Query engine-t (DocValues bevezetése, régi lekérdezés-építők tiltása, átszervezett analizátorok)

[mcserep]

@LakatosMark1 Thanks for your results!

We use the English language on GitHub for the project. Please update your remark accordingly.

[LakatosMark1]

Excuse me:
Lucene 5.5.X: Only requires replacing imports and version constants, no need to rewrite the database schema, and old indexes can be preserved
Lucene 6.6.X: Minimum Java 8, old number handling (NumericField, IntField) is permanently discontinued, new PointValues ​​and DocValues ​​logic must be used
Lucene 10.X: Java 21: entire indexer and Query engine must be rewritten (introduction of DocValues, disabling old query builders, reorganized analyzers)

[mcserep]

@LakatosMark1 As discussed today, we shall proceed with smaller steps. Try to upgrade to version 6, as that task shall fit in the timeframe for this semester.

[mcserep]

@LakatosMark1 Additionally, as discussed on today's meeting, please investigate the UTF-8 / UTF-16 parsing issue. As far as I remember, originally it was reported when parsing C# code with such characters. Create a simple, Hello World program in C#, but e.g. use emojis as variable names, as that is permitted in C#. Try to parse it and observer if there are any issues?

If there is a problem, but not related to the search parser, we will create a separate GitHub ticket for it, but first it should be analyzed in more detail.

[LakatosMark1]

@mcserep I have made a pull request for this issue, it solves the search parser error.