MajSimai is an interpreter for Simai, written in C#.
- Basic metadata fields
- Custom simai commands with '=' as separator
&commandPrefix=value
- Comments
||Hello world!(to the end of the line, only inside&inote_Nchart text) - Time signature
||s3/4 - BPM declaration
(float) - Beats declaration
- Common
{int} - Absolute time
{#float}
- Common
- Tap
1- Break flag
1b - EX flag
1x - Mine flag
1m - Force star flag
1$- Fake rotation
1$$
- Fake rotation
- Break flag
- Hold
1h[duration]- Short form
1h - Break flag
1hb - Mine flag
1bm - Duration format:
- Common
[division:beats] - With absolute time
[#time] - With custom BPM
[bpm#division:beats]
- Common
- Short form
- Slide
x-y[duration]- Same head
1-3[duration]*-7[duration] - Break flag
1-3[duration]b1-3b[duration]
- Mine flag
1-3[duration]m1-3m[duration]
- Conn slide
1-3-5-7[duration]1-3[duration]-5[duration]-7[duration]
- No head slide:
-
1?-3[duration] -
1!-3[duration]
-
- Tap head slide:
1@-3[duration] - Duration format:
- Common
[division:beats] - With custom BPM
[bpm#division:beats] - With custom BPM and absolute slide length
[bpm#time] - With absolute slide start time and common slide length
[time##division:beats] - With absolute slide start time and absolute slide length
[time##time] - With absolute slide start time and custom BPM length
[time##bpm#division:beats]
- Common
- Same head
- Touch
B1- Hanabi flag
B1f - Break flag
B1b - Mine flag
B1m - Ex flag
B1x
- Hanabi flag
- Touch hold
B1h[duration]- Short form
B1h - Hanabi flag
B1hf[duration] - Break flag
B1hb[duration] - Mine flag
B1hm[duration] - Ex flag
B1hx[duration] - Duration format:
- Common
[division:beats] - With absolute time
[#time] - With custom BPM
[bpm#division:beats]
- Common
- Short form
- Each note
note/note- Fake each
1`2`3`4,
- Fake each
- EOF flag
E(anEwhere a note starts that is not a sensor such asE1) - Slide Code
1A357K1[2:1](MajSimai docs)
To use MajSimai in your own project, you will need to add a reference to the MajSimai library in your solution.
// Specify the chart file
var filePath = "/path/to/your/chart.txt";
using var fileStream = File.OpenRead(filePath);
// Parse into SimaiFile
var simaiFile = await SimaiParser.ParseAsync(fileStream);
// With special encoding
simaiFile = await SimaiParser.ParseAsync(fileStream, Encoding.ASCII);
// Or parse only metadata
var simaiMetadata = await SimaiParser.ParseMetadataAsync(fileStream);
// With special encoding
simaiMetadata = await SimaiParser.ParseMetadataAsync(fileStream, Encoding.ASCII);
// Or parse from the chart text you provide
var fumen = "(197){8}1,1,1,1,1,1,1,1,2b,3h,4-1[8:1],";
var simaiChart = await SimaiParser.ParseChartAsync(fumen);Since 3.0, a slide is read as its star head plus its slide paths (issue #12):
- A slide with a head, such as
1-5[8:1], becomes aTapnote followed by aSlidenote. TheTapis the star head: itsRawContentis the text before the first slide mark (1), andRelatedNoteslists its slides. - The later parts of a same-head group (
1-3[8:1]*-5[8:1]) are also listed in theRelatedNotesof the head. - A slide without a head (
1?-5[8:1],1!-5[8:1]) is a singleSlidenote. - On a
Slidenote,IsBreakandIsMinedescribe the slide path (they wereIsSlideBreakandIsMineSlidebefore 3.0).IsSlideNoHeadwas removed.
Every note also has these flags:
| Property | Meaning |
|---|---|
IsEach |
A slide is an each when its timing point has more than one slide. Any other note is an each when its timing point has more than one note that is not a slide. |
IsStar |
A Tap that uses $, or that has related slides and does not use @. |
IsStarDouble |
A star with related slides whose timing point has more than one slide, not a mine, starting from the same button. |
SimaiChart.CommaTimings is created the first time it is read, which saves most of the memory of a parse when it is
not used.
Every Parse* method takes an optional SimaiParserOptions:
// Lenient (the default): reads charts the way MajSimai always has
var file = SimaiParser.Parse(stream);
// Strict: throws for every note that cannot be read and every text the grammar does not define
var strictFile = SimaiParser.Parse(stream, SimaiParserOptions.Strict);| Lenient | Strict | |
|---|---|---|
A declaration that is not closed, or whose value cannot be read; a single | |
throws | throws |
| A malformed note (see below) | skipped (Warning log) | throws |
A note that cannot be read, such as 1-5 without a duration |
skipped (Debug log) | throws |
Note text cut off by a declaration, 1(120)2, |
1 dropped (Warning log) |
throws |
| Note text without a comma at the end of a chart | dropped (Warning log) | throws |
Stray characters, empty / or ` parts, b / m in the middle of a slide path, a malformed ||s signature |
ignored | throws |
Text after the end marker E |
ignored (Warning log) | throws |
Note shapes: buttons 1-8, sensors A1-E8 / C / C1 / C2, hold and touch layouts |
not checked | checked |
| Slide paths: end buttons allowed for each shape, durations, Slide Code | not checked | checked |
& line without = / unreadable &first |
ignored (Warning log) | throws |
| A chart that fails in a file | kept with its text and no timing points (Error log) | the whole parse throws |
Exceptions:
InvalidSimaiSyntaxException(both modes): an unclosed(,{or<declaration, or a BPM, beat or HS / SV value that cannot be read.InvalidSimaiMarkupException(both modes): a single|.InvalidSimaiNoteException(strict mode only): a malformed note, that is a note with nothing left after its markers (1/b), a touch area letter that is not at the start of a note (12E,1-5E[8:1]), an empty*part (1-5[8:1]*; the other parts of the group are kept), a duration with too many separators ([8:1:2],[1#2#3#4#5]) or with text between#([3#9#1.5]), or a hold whose]comes before its[. Lenient mode skips these notes and logs aWarning. Strict mode also throws it for every other note rule in the table above.
The position of an exception is the line and column of the declaration, or of the start of the timing point that holds
the note. Numbers are read with the invariant culture; exponents, NaN and Infinity are accepted, thousands
separators are not.
SimaiSyntaxChecker reports every problem of a file or chart instead of stopping at the first one:
foreach (var diagnostic in SimaiSyntaxChecker.Check(File.ReadAllText(path)))
{
Console.WriteLine(diagnostic); // inote_5 34:6: Error SM301: The slide "1-2" in "1-2[8:1]" cannot be drawn: ...
}
var chartDiagnostics = SimaiSyntaxChecker.CheckChart("(120){4}1-2[8:1],");An Error breaks the grammar: a text parses in strict mode exactly when it has no errors. A Warning is valid but
questionable. The checks include:
- buttons, sensors and the layout of every note;
- the end button of every slide shape, following the official simai table (for example
-needs 2 to 6 buttons between start and end,s,zandwonly end at the opposite button,Vturns 2 buttons away);1v1is a Majdata extension and only gets a warning; - connected slides: durations on every segment or only on the last one, and no
wsegment; - Slide Code: command syntax and parameters, a
Kend, and connections that cannot be drawn (a node inside its circle,Pdirectly toQ, circles 0 and 9); warnings for zero-length moves and three or more turns around one circle; - fields set twice,
&lines without=, and&firstvalues that are not numbers.
Codes are listed in SimaiDiagnosticCodes: SM1xx chart structure, SM2xx notes, SM3xx slide paths, SM4xx
fields.
MajSimai writes messages through SimaiLogger.Current. The default logger writes Warning and Error messages to the
console. In lenient mode a chart that fails to parse does not fail the file: it keeps its text, has no timing points, and is
logged as an Error. Text that is dropped is logged as a Warning, and notes that cannot be read as Debug.
// Show everything, including notes that cannot be read
SimaiLogger.Current = new ConsoleSimaiLogger(SimaiLogLevel.Debug);
// Or forward to your own logger by implementing ISimaiLoggerdotnet run --project Tests/MajSimai.TestsThe tests compare the lenient parser with the published MajSimai 2.2.3 package (downloaded from nuget.org at restore),
on unit cases and on generated charts; the intended fixes listed above are aligned before comparing. To also compare every maidata.txt of a folder, download the samples and set
MAJSIMAI_SAMPLES_DIR:
python tools/download-samples.py "Simai Test Samples"
MAJSIMAI_SAMPLES_DIR="Simai Test Samples" dotnet run --project Tests/MajSimai.Tests -- -class "*SampleEquivalenceTests"The sample tests also check that the syntax checker reports an error for exactly the files strict mode rejects, and print how often each diagnostic occurs.
Set MAJSIMAI_BENCHMARK=1 as well and run -class "*ParserBenchmarkTests" in Release to compare the speed of both
parsers.
tools/MajSimai.ManagedBenchmark measures the parser with BenchmarkDotNet, on the
samples named by MAJSIMAI_SAMPLES_DIR (or by MAJSIMAI_BENCHMARK_CORPUS, a corpus file written by
tools/benchmark-native.py). It takes the options of BenchmarkDotNet and runs every benchmark when no --filter is
given:
MAJSIMAI_SAMPLES_DIR="Simai Test Samples" dotnet run -c Release --project tools/MajSimai.ManagedBenchmark -f net10.0 -- --filter "*FileParseBenchmarks*"
MAJSIMAI_SAMPLES_DIR="Simai Test Samples" dotnet run -c Release --project tools/MajSimai.ManagedBenchmark -f net10.0 -- --filter "*FileParseBenchmarks*" --runtimes net10.0 nativeaot10.0| Benchmark | One operation |
|---|---|
FileParseBenchmarks |
Parses one sample, the median, 99th percentile or largest by size (Sample) |
CorpusParseBenchmarks |
Parses every sample. An operation takes seconds, so add --strategy Monitoring --warmupCount 1 --iterationCount 5 or let BenchmarkDotNet choose the counts |
Each class has two methods: Parse (SimaiParser.Parse, the baseline) and ParseToNative (MajSimai_ParseWithTolerance
and MajSimai_Free called in-process: the parse and the copy into native memory that the C++ wrapper asks for). The
memory diagnoser reports the managed memory allocated per operation. The benchmarks install a logger that keeps the
levels of the default logger, because some lenient checks run only when their level is logged, but writes nothing.
--runtimes nativeaot10.0 needs the native AOT prerequisites (on Windows, the Visual Studio C++ build tools).
To compare the speed of the cimai C parser, the C# parser and the C++ wrapper on the
samples, run tools/benchmark-native.py. It makes the package with tools/build-cpp-package.py (or takes
--package), builds tools/NativeBenchmark with CMake (cimai from the sources of --cimai), runs each native parser
over the samples in memory in its own process, and runs CorpusParseBenchmarks with BenchmarkDotNet on net10.0 with
the same warm-up and measured runs:
python tools/benchmark-native.py --cimai path/to/cimai --samples "Simai Test Samples" --kind shared --iterations 10| Run | What is measured |
|---|---|
cimai |
cimai_parse and cimai_file_release |
majsimai-cs |
SimaiParser.Parse on the .NET runtime, called the way the native entry point calls it |
majsimai-cs-native-abi |
MajSimai_ParseWithTolerance and MajSimai_Free called in-process on the .NET runtime |
majsimai-cpp-view |
SimaiParser::ParseAsView: the parser compiled with native AOT, and the copy into native memory |
majsimai-cpp-model |
SimaiParser::Parse: as above, then a copy into std::string and std::vector |
The table and report.md/report.json in artifacts/benchmark/<kind> give the median time of a run over every file
and the throughput; the native rows add per-file percentiles, the C# rows the memory allocated per run (BenchmarkDotNet
writes its own reports to artifacts/benchmark/<kind>/bdn/results). --parsers picks the runs (the C# runs need
neither CMake nor cimai) and --limit N uses only the first N files for a quick check. On Windows the native runs need
the Visual Studio C++ build tools; CMake is taken from the path or from Visual Studio.
The C++ wrapper tests compile Bindings/Cpp with the compiler in CXX or the first of c++, g++ and clang++ on the
path, and are skipped without one; they also check that code misusing the owner of a result does not compile. To also
link a package with CMake and compare its results with the C# parser in lenient and strict mode (with
MAJSIMAI_SAMPLES_DIR set, on the first 200 samples as well), make a package and set MAJSIMAI_CPP_PACKAGE:
python tools/build-cpp-package.py --kind static
MAJSIMAI_CPP_PACKAGE=artifacts/cpp/linux-x64-static dotnet run --project Tests/MajSimai.Tests -- -class "*CppBindingNativeTests"If you want to compile MajSimai yourself, please make sure that the .NET 10 SDK (or newer) is installed on your computer: MajSimai is written in C# 14.
The target frameworks available for MajSimai are as follows:
netstandard2.1net5.0net6.0net7.0net8.0net9.0net10.0
Then, use the following method to compile it:
git clone https://github.com/LingFeng-bbben/MajSimai.git
cd MajSimai
dotnet restore
# Feel free to change the target framework
# If you want to generate nuget package, please remove /p:GeneratePackageOnBuild=false
dotnet publish -c release /p:PublishAot=false /p:GeneratePackageOnBuild=false -f netstandard2.1
MajSimai is AOT and Trim compatible.
If you want to compile native library, you need to specify the use of AOT.
dotnet publish -c release /p:PublishAot=true -f net9.0 -r linux-x64In the native library, MajSimai exports these functions (declared in Runtime/Unmanaged/UnmanagedSimaiParser.cs):
MajSimai_Parse(const char* content, int32_t contentLength, UnmanagedSimaiParseResult* result)parses a file in lenient mode. The content is UTF-8 (a UTF-8 or UTF-16 byte order mark is honored).MajSimai_ParseWithTolerance(const char* content, int32_t contentLength, SimaiSyntaxTolerance tolerance, UnmanagedSimaiParseResult* result)parses in lenient (0) or strict (1) mode. In strict mode the first chart that breaks the grammar fails the parse withCODE_INVALID_SIMAI_SYNTAXorCODE_INVALID_SIMAI_MARKUP, and the line and column of the error are counted in the text of that chart.MajSimai_Free(UnmanagedSimaiParseResult* result)frees a result, whether parsing succeeded or not, and returns 1 (0 for a null pointer).MajSimai_FreeHGlobal(intptr_t ptr)frees a block that MajSimai handed over to the caller and returns 1 (0 for a null pointer). No function hands memory over at the moment, so it is never needed for a parse result.
Ownership: the caller owns each result and frees it with MajSimai_Free exactly once. Everything the result points to
belongs to the result: never free those pointers, never keep them after MajSimai_Free, and never free two copies of
one result. The result passed to a parse function is overwritten without being freed, so it must be zero-initialized
or already freed. The signatures only use blittable types (int32_t instead of bool), so they work without
DisableRuntimeMarshalling.
The native layouts changed in 3.0: notes follow the 3.0 note model, related notes are indices into the notes of the same timing point, and every text is UTF-8 with its length in bytes (2.x used the ANSI code page with lengths in UTF-16 characters). The native library logs through the default logger, so its warnings go to standard output.
Bindings/Cpp is a header-only C++17 wrapper generated from the native ABI by tools/MajSimai.CppGen, and each release
publishes it with a static library per platform to MajSimaiCpp. The
README of MajSimaiCpp documents the API, the packages made by
tools/build-cpp-package.py, the release workflow and the rules of the ABI. Regenerate the wrapper after changing
Runtime/Unmanaged; a test fails while Bindings/Cpp is out of date:
dotnet run --project tools/MajSimai.CppGen # writes Bindings/Cpp
dotnet run --project tools/MajSimai.CppGen -- --check # exits with 1 when Bindings/Cpp is out of date