Neo Geo 240p Test Suite Status and MDFourier Analysis tool

First of all, I wanted to thank you for your support. Making this would not be possible without your generous help. It helps me greatly.

I also want to apologize because I am terrible at reporting my progress... once I'm programming and measuring stuff I want to continue doing so, until I'm finished. I get carried away.

I will post a small video detailing the current feature set for Neo Geo, and want to do a live stream with Dasutin presenting this new release. Please comment if you'd like to watch that.

Neo Geo progress

I didn't post the advance of the suite last month but I've been working all my free time on it. It is basically feature complete and in a release candidate status for AES, MVS and Neo Geo CD, all tested in real hardware

The z80 sound driver was coded in assembly during this month, and a feature complete version of MDFourier running on it using FM, SPSG, ADPCM-A and ADPCM-B and is being tested for appropriate patterns across hardware revisions.

The change log is massive since it has been built from scratch and up to the standard of mature versions in other platforms. Extensive tests have been made so that the quirks of each platform are taken into account, such as frame rates, UniBIOS behavior on each, differences in resolution (224 vs 256 in NTSC/PAL, 304 horizontal official vs 320 as the hardware supports, in all patterns)

Audio in the YM2610 was a big chunk of work this month, since I had been putting of writing the sound driver in z80 assembly. I shouldn't have. It was really fun and I learned a lot, which I want to put in practice for other Z80 audio drivers in other platforms. I hope to get you some news on that this year. Not to mention that I had to learn a lot about the Neo Geo hardware details to have it making all I wanted in Neo Geo CD and UniBIOS variants as well.

I also had to patch MAME to get the UNIBIOS to work with the Neo Geo CD, and aso got the Front Loader and Top Loaders to boot in MAME, but with some glitches even with original BIOS. I will clean them up a bit and send a pull request, or at least post it for some of the developers to use the findings. This was possible because I had been checking how the BIOS uses some memory mapped registers on real hardware with the Register Viewer. And simply matching the differences with the MAME driver made them work.

Even if Neo geo details were what I worked the most, and what I have more to say about, I will reserve those for the live stream after/during release very soon. 

And yes, the Neo Geo has HCFR support. I believe that its color capabilities are decent enough for this use, even if limited when compared to Dreamcast. testing will be underway so that proper documentation can detail its limitations when using this as such.

Also... would you be interested in a Physical release of the Neo Geo version? This could be a possibility of there is enough interest and our tests go well. Of course there are tons of issues in that avenue to resolve, production, distribution.. I rather not think about it...

MDFourier Analysis tool

I spent a week this month with Neo Geo development paused since there were some suggestions and reports for the MDFourier analysis tool, by Bernard and one by sajattack. The Neo Geo ADPCM comparison also triggered a few new requirements that matched what they reported, and I believe the results are really good and will improve accurancy and flexibility across all systems.

Time Domain plots

First sajattack was working on measuring the Commodore 64 for MiSTerFPGA. He brought to my attention that the Time domain plots of the referenced were scaled when the comparison had amplitude differences. This was confusing because he thought that the changes he made in the comparison signal while changing parameters were not having the desired effect. 

This was done on purpose in order to keep both signals at -3 dbfs in the Time Domain plots... but after he reported it and I saw what was causing the confusion, I noticed that my decision violated one of the main axioms of MDFourier's design: "The reference is always a reference". It is always "right" and everything measured should be relative to the reference file.

I decided to change it so that the comparison plots could now clip at 0dbfs and show them at the reference signal scale. This unexpectedly was in a collision course with other fixes....

Noise floor, digital signal decay, time variation and Power Spectral Density

I had been dreading tackling an issue regarding Noise floor measurement of digital sources, which Bernard brought back to my attention. His suggestion matched what I had considered before, but having a fresh and renewed view after not working on the analysis tool for a few months pushed me to tackle it.

The basic idea was to cut off the frames which contained the decay of the pulse tones or audio signal, which usually go down to -126 db in the noise floor, when a digital source captured digitally should have none of.

The problem with doing so is that all profiles, the set of instructions which tell MDFourier how to cut and process the signal, have attempted to balance the lengths of each segment to analyze. When you compare audio chunk of different with a Discrete Fourier Transform the amplitudes won't match relatively between the blocks, since the power and frequency bin distribution will be different. 

From the start this had been compensated carefully by two processes. the first one discards sub sample time matches to always have the signals aligned. And the second one created windows that masked the extra time so that we'd have apples to apples comparisons in both frequency and amplitude, aside the frequency domain normalization.

I understand some might be horrified by this notion, but in reality it modeled the uses cases we had perfectly. This works because in most game consoles we had covered the audio of the system is clocked independently from video. In other words, a PAL Mega Drive audio "note" would last longer but the exceeding length would not be present in the NTSC Mega Drive, and thus masking the extra time via windowing results in a perfect match.

Here are a couple of images that illustrate it:

These are created by the Time Domain plotter (accessed via -uu) and show how the signal is being processed, the red block in the PAL signal will be masked and as you can see, they magically match.

However, this is not always the case. There are computers and game consoles that do vary the audio signal when the CPU or video signals have different clocks. And that was giving us inaccurate results.

Of course the Commodore 64 that sajattack was comparing falls into this particular category, and thus both threads of changes got intertwined. I now needed to find a way to compare both signals when they might have different time lengths.

I spent a couple of days trying to compensate the normalization options by experimenting with artificially created signals that perfectly demonstrated the issue.in a very controlled manner. The "old" windowed option worked perfectly for its own scenarios, but the newer profile option that allowed this new type of comparison by zero padding (and thus aligning the frequency bins) was not matching in amplitude. I believed that the code/algorithm had to be wrong. The experienced Signal Processing reader might already know the answer and might be smiling and shaking their head (spoiler, it wasn't wrong... not completely at least).

I understood what was wrong in my reasoning while taking a shower. of course it had to be there. OBVIOUSLY if the time chunks I was comparing with the same signal in them were differently in length, amplitudes would never match. By definition they shouldn't! But I was confused because I was dealing with two issues at once... the above issue of matching the amplitude in chunks of the noise floor against the signal when they had different length and I wanted the amplitudes to relatively match properly, and the issue of comparing a 20ms vs 16ms lapse in PAL vs NTSC signals. But looked like the same, but they were completely different. However the solution to both was the same, i was just checking the consistency of my solution to one method with the verification test of the other.... because they are so similar in nature. Dumb me.

I found this blessed paper.

And I knew that was the precise solution to my problem.

So I quickly implemented it and compared teh results with  my artificially generated signals, and after they gave the exact results I had predicted when designing the signals, I went to real signals. 

And it worked. 

Both problems solved. If you have reached this point, please share a comment dear reader that another soul took this ride with me. It had me sleepless for two or three days while testing and comparing stuff, you know... going to sleep with such an issue in your mind only gives you dreams where you try to solve it until you wake up... tired. Star Trek First Contact was an interlude I took in order not to go to sleep thinking about this. And it did help that night.

So far i stand by these results, and of course comparing a PAl vs NTSC signal with such characteristics will not match as neatly as a PAl vs NTSC Mega Drive, since the clocks differ the frequencies produced will also change in pitch. It is so obvious now, as it was to me when I coded it originally a few years ago. But verification and proper Amplitude readings were of utmost importance.

Aside form these, other changes regarding scale for Time domain plots were fixed when samplerates didn't match between signals, etc. I would have to read teh changelog to remember the rest, but these were what took me from Neo geo development for a week.

Right now these changes are in a release candidate being tested before release.

Sega CD PCM RAM tests

Today I changed the Sega CD PCM test a bit to better work on Laseractive systems thanks to Leo Oliveira testing and suggestions. The BIOS had some behaviors that differ from other Sega CD systems, and they have been accounted for.  If you need these before I release the updated ROM (I usually wait for a few changes to accumulate), please contact me.

Thanks for your interest and attention reading this, and for your help and support.