MDFourier precision improvements and X68000 CLI Beta

I know, I have to yet release the monoscope and Donna for the remaining platforms. Wii and GC are almost there, I have just been changing the interface to adjust to the newer menu style and have to add a controller test. But Keith Raney has already validated the monoscope PAL and NTSC patterns and they have been integrated. If you need the beta, please ask for it, I will push that release out this month though.

Genesis still needs a PAL monoscope pattern though, so that will take a bit longer. PCE already has most changed made, however some modifications might arise, hence why I've not released it yet.

Speaking of Donna, check out the new art made by José Salot:

There are prints at his site if you want to have one and support his amazing work.

MDFOURIER on the X68000

I worked on a version of MDFourier for the X68000 during the first half of the month, testing it on real hardware and learning how to use the OKI and YM2151 to produce sound.

I did find there was an example on how to produce ADPCM in the "Inside x68000" book, and decided to test it out. Of course it needed transcription, so I released that at github. 

With that taken care of, I started work on creating tones for the YM2151 and also having a cross compiler working for x68000. The development cycle is now done via a NULL modem serial cable so I can test on real hardware, as usual.

In order to understand that better, I made a VGM decoder and player for X68000, that was mainly academic but helped a lot, I loved listening to Dragon Spirit's first stage theme played from the format by my code.

Here is an example MDFourier plot from the current version. I hope it will be useful for the amazing MiSTer FPGA project, and any other emulation or check for people interested in the platform. I hope to be able to release that this month as well, even if only as a command line tool at the time being. I would love to work on a full 240p test suite version for X68000, and maybe I will since I tend to love working on these platforms.

MDFourier improvements

The later part of the month I was obsessed with improving MDFourier. There have been several things that needed improvement or fixing, that have been barriers for me to release that mythical Audio Equipment test made by Pinobatch. 

Noise Floor

The first one was making the Noise Floor graphs show as much info as possible. Those plots have been subject to the rest of the system parametrized options, including the frequency count depth - a parameter that limits how many frequencies are compared per block/note. The default in MDFourier is 2000, which is a rule of thumb value that worked during testing for speed and having a lot of information. That's 2000 frequencies compared for each 150 to 300milliseconds from a signal, per stereo channel.

After the change, no info is discarded and of course it depends on Nyquist and samplerates, but that is easily 10x more now. 

This was noticeable when comparing 192khz recordings using the full range, there were "holes" in the noise floor graphs caused my data that was discarded due to amplitude or slots. Here's how previous versions displayed that information in such extreme cases:

And this is how it is displayed now. 

Thanks a lot to grievre from the MDfourier team for pointing this out.

Sync Alignment

And now, to the deep changes. When using the Audio Equipment test, which is brutal, sync alignment is crucial. Sync alignment is the internal process in MDFourier that finds out a pattern of tones at the start and end of the signal, and uses those to determine the speed at which the recorded hardware was playing or, any deviations in the audio interface since both could add up to the same discrepancy in timing from the recorded signal.

The process is crucial since it allows MDFourier to align both signals and segment them for comparison, hopefully at around the exact sample it is needed. When a discrepancy is found, that is one or both signals have a timing difference by either time dilation or time expansion, the algorithm has to compensate for that and measure each at their own speed but having them be equivalent in length after a DFT (Discreet Fourier Transform) is performed. 

During the first days of November it occurred to me that I could perform a secondary step after sync detection: take the first pulse of such sequence and measure its length. Then locate the middle and place that aligned to the middle of the expected duration, and then verify the alignment by doing small incremental FFT of just a few samples in order to match both better. And the result was really, really good. It found differences of around 4 samples in a 192khz signal, which could be verified manually on a perfect digital audio file. That was impressive.

This lead me to finally be able to tackle the Audio Equipment test inconsistencies I had found earlier this year. You see, I added a 1050TI GPU to my desktop for video rendering, and after that I noticed some of my recordings started to show terribly in MDFourier under the Audio equipment Test. This immediately lend me to believe it was a power supply issue, and I changed it. But, the plots were still off. 

To make a long story short, the power supply was the real issue, but it was not the only one. The precision of just a few samples being off was another cause, which I could only fix this month.  But there was a third issue at hand, and that is that I had introduced a bug in MDFouriera few months prior that only shows at extreme cases like this. 

I mentioned above that MDFourier has an internal mechanism to compensate for small discrepancies, well it has two. One that compensates for the difference in timing between both signals, and the other that deals with sub-sample compensation. You see, if the comparison calls for a 0.333766 seconds chunk to be analyzed, that would translate to 16020 stereo samples at 48khz, however it is not exact, it actually is 0.333750 seconds, that is a 0.000016 difference which anybody would say is ridiculous, and it is. But these add up and signals get misaligned quickly. 

This is why MDFourier keeps track of a fractional part of a sample, sub sample counter, that when added up reaches a sample it gets skipped and everything gets in sync back again.

This process is crucial for MDFourier, and I had introduced a bug into it when I changed the internal structure of the software around at the same time those two other issues were there.

But, it is now fixed.. although I should say partially fixed to be honest with you dear friend who has reached up to this point in the post. It is completely fixed in one direction, that is when one of the signals needs frames dropped. But it isn't in the other direction yet, at least not to my satisfaction. In reality it is working surprisingly well, but I either don't fully understand how it works both ways when I just designed it for one scenario, or it still has a very small error which doesn't show visually in all tests I've thrown at it. But in either case, I will be getting to the bottom of it, hopefully this week. I made some artificial extreme cases to simulate what I was seeing but under controlled circumstances. I created a version of the Audio Equipment Test at 192Khz that is off by 1hz either forward or backward, that is it is really at 192001 hz or at 191999 hz. These are the kind of test scenarios which I like to throw at this to confirm what I see on the field. Here are the current results from both

At 192001hz

And at 191999hz

For the record, this is how 191999 showed before fixing it:

While taking a walk today I believe I have found the conceptual and implementation inconsistency and will be testing this soon.

Speed improvements

Again thanks to grievre for suggesting Open MP as a solution to use more cores when creating the plots. This changes have been in the internal betas for a while and I also made a huge improvement to the algorithm that generates the spectrograms, which should reduce the wait by 40% in some cases.

Aside from that, The Makefile now autodetects the supported platforms so a simple make command should work on linux, macos, Msys2 and cygwin. I hope to finally make a public release with this and a year worth of changes to MDFourier, deep down I knew something was off and these fixes have given me confidence that everything is back to the precision it should be.

I also added support for IEEE 64 bit PCM WAV files and 8 bit PCM files just for fun.

Thanks a lot for your support, it means a lot to me. And I will forever thank you. Please take care.