Maxon

Hi. I started making a plugin, ((X)) Surround System, last weekend that seems to be upmixing 2ch audio in a fashion reminiscent of Dolby Pro Logic II. I'm not totally sure if it actively matrixes anything, but I'm sure someone else can figure that out.

Here are the sources that I have been using while developing this plugin:

http://www.dolby.com/uploadedFiles/z..._Operation.pdf

Dolby Pro Logic - Wikipedia, the free encyclopedia

I would say that it has been coming along well, but I can't figure out how to correctly apply a matrix. I am also looking for a better way to compare the status of the different channels so that the separation will be better, and I can fully remove voices from the rear channels, whether they are in the center, left, or right channel of the stereo track. For example, if you listen to a song, particularly Basket Case by Danger Doom, where someone is saying something only from the right channel, it will playback in the rear right with a distinguishable level of prominence while playing in the front at the same time.

The code is below:
-------------------------------------------------------------------------
; Registers
input in_L, in_R;
output L, R, Ls;
output Rs, C;
static C_constant=0x5a827999, SL_phase=0x7fffffff, Sr_phase=0x80000000;
temp SL, Sr, offsetrL;
temp offsetrR

; Code
macs C, in_L, in_R, C_constant;
macsn SL, in_R, in_L, 0x7fffffff;
macs Ls, offsetrL, SL, SL_phase;
macsn Sr, in_L, in_R, 0x7fffffff;
macs Rs, offsetrR, Sr, Sr_phase;
limit SL, Sr, L, Ls;
limit offsetrL, 0x80000000, Rs, offsetrL;
limit Sr, SL, R, Rs;
limit offsetrR, 0x80000000, Ls, offsetrR;
macsn L, C, in_R, 0x20000000;
macsn R, C, in_L, 0x20000000;

end
-------------------------------------------------------------------------

The code is from the sixth edition that I have made; I can post the earlier versions if anyone is interested. I would also prefer to keep it ultralight by removing anything that it doesn't need to be functional that can be taken up by another plugin, like speaker delays.

Here are some of my sources for testing its ability to upmix 2ch audio:
YouTube - Killzone 2 Gameplay HD 720P Widescreen HD Quality
DiRT Video Game, Dusty Trails Gameplay HD | Game Trailers & Videos | GameTrailers.com
YouTube - Gran Turismo 5 Prologue: Ferrari F40 replay
YouTube - Sonic Unleashed Trailer "High Europe" Spagonia
YouTube - Gran Turismo 5 Prologue: Ferrari F40 replay
YouTube - Ferrari F40 testdrive by FifthGear
YouTube - Soul Calibur IV Hilde And Sophitia Gameplay
YouTube - Project Gotham Racing 4 - Corvette HD
YouTube - Project Gotham Racing 4 (PGR4) [Tokio]
and Computer games like NFSU2, GRID, CoD 4 etc.
Footage from movies and television like the Fifth Gear test drive should be usable.

For the record, my knowledge and understanding of this kind of stuff is very limited, and I don't fully understand the instructions.
I also don't know if this guide covers all of them.
kX Project -- A Beginner's Practical Guide to Programming Digital Audio Effects on the E-MU10kx family of DSPs in the kX Project Environment
As a result, anything that can help me understand them will be valuable to me.
Thanks!

Max M.

>As a result, anything that can help me understand them will be valuable to me.

well, there's nothing complicated there really. In fact one should start with understanding the only instruction: macs, as multiplication and addition cover 99.9% of signal processing algorithms. The rest is just a matter of understanding the effect algorithm and the math behind it. And this (the algorithm and the math) is what i would recommend you to dig into first
- start with writing the algorithm math on a paper in some pseudocode (whichever "pseudo" you feel better with - for example: "a = b + c + d * e - f * g") and then convert the stuff into dane code. You can always ask us here for the details on a particular instruction and its use in a certain algorithm chunk - but first, the algorithm itself, algorithm, algorithm, algorithm!

__________________

Maxon

Hi again. Thanks for the advice; I got the matrixing right this time . However, in order to keep the sounds from the left and right channels that should be playing in the front, like dialogue on the left or right, from playing in the surround speakers, I need to be to make a voltage-controlled amplifier that will lower one channel and raise the other when sounds like those play. I am assuming that making the volume multipliers for the Ls and Rs channels inversely proportional would work, but I'm not sure how to do it. Is there a way to use the MACS function so that the R operand is constant without using the constant register? I tried using the constant register, but it can't be used as R in the function.
Thanks again.

Oh yeah, I need to be able to do something like this in order to set up the L and R channels in stereo. Otherwise, they play sounds that should only be coming from the back.

Max M.

>I am assuming that making the volume multipliers for the Ls and Rs channels inversely proportional would work

Nope. Don't miss the important detail - the VCA should be controlled not by the signal waveform (what you're trying to do) but by the signal envelope - which you need to extract first. Or, in other words - the "balancing servo" engine of the active matrix is just a kind of dynamics processing, or yet in another words, simply a form of compressor/expander. To give you more keywords for search (algorithms! ;): "envelope detector + vca".

>Is there a way to use the MACS function so that the R operand is constant without using the constant register?

No, you cannot "invert" an instruction operation by "swapping" the meaning of its operands.
Well, the "inverted" multiplication could be a division or (depending on the goal) inversion of multiplier itself (like a = b * (1 - c))
- but this is not what you really need (at the moment). The sample-per-sample multiplication/division of one waveform by another will give you just a distortion not a "level control".

----
I guess your next turn is to dig into "compression algo" articles/papers ;) (like this for example)

__________________

Maxon

I'm trying to get the first envelope detection method mentioned above working. When I play it back as an output signal, it comes out distorted.
http://www.mathworks.com/products/demos/shipping/dspblks/dspenvdet.html#7
Also, how do I get a visual confirmation of the detection?

Max M.

well, you can assing the output of the detector to an extra output of the plugin and then analyze it the waty you like (for example record in your favorite editor and check visually)

__________________

Maxon

Forget what I said; I've got it now. I'll probably work more on this next week after I've taken all of my finals.

I got the envelope detection working for the left and right totals. If I'm correct, I should have to detect the envelope for the L+R and L-R signals as well, but that should be a later concern. Anyway, I'm still not sure how to go about setting up the VCA in its entirety.
Variable-gain amplifier - Wikipedia, the free encyclopedia
Voltage Controlled Amplifier
SSM2018 | Trimless Voltage Controlled Amplifier | Operational Amplifiers (Op Amps) | Amplifiers and Comparators | Analog Devices
4QD-TEC: Voltage controlled amplifier circuit
I did some searching, but all I can find are circuit diagrams. I'm assuming that I am going to have to understand them before I can finish this plug-in. This would also mean that I will have to study electronics up to a level appropriate for this.
Would these be good places to start?
Electronics: An Online Guide for Beginners - Schematic Diagrams
Tutorial - How to Read a Schematic
The Physics Classroom
Learn Physics Today!
Electricity & Magnetism

Max M.

That's not the hardest part of it. You just use your envelope (but after applying appropriate attack/release timings and necessary scale, that's the trick - see below) directly as VCA multiplier, e.g. it's just: "output = input * voltage_control" (and this single line is the complete VCA module).
The tricky parts are:
1. What timings you will need to apply to your envelope before VCA (e.g. how fast changes in envelope level will affect corresponding output level changes) - assuming the "active matrix" is somewhat similar to "automatic level normalizer" - that would be relatively short attack and pretty long release.
2. How deep (e.g. the "scale") and in what degree the envelope level should affect corresponding output - e.g. you need to convert your initial arrangement (... "to compare the status of the different channels so that the separation will be better, and I can fully remove voices from the rear channels, whether they are in the center, left, or right channel of the stereo track...") into the math. E.g. sorta "which levels of which channel to which 'voltage_control' and how".

Etc.

I don't think any VCA circuits will shed a light on that - just because the magic of an "active matrix" is really outside the VCA. Have you read those tutorials/articles about compression and dynamics processing ("compression" as in Dynamic range compression - Wikipedia, the free encyclopedia, Harmony Central - Compression/Limiting etc. etc.)? I believe that should give to you a lot of hints on the math behind the stuff even if there isn't any math equations or code.
As for the exact layout and fine-tuning of all those active-matrix modules (envelope detector, timing filters, vca), well, i guess only a complete documentation and/or source code for a "dolby-like" active-matrix can provide precise direction - but i'm not aware of such documentation/code. So it's going to be total trial-and-error.

After all, i'd said that the "active surround matrix" is quite complex thing to make it completely from scratch - as there's no common convention for such processing and the algorithms of proprietary processors are completely undocumented. So well, perhaps, first i'd suggested to start with a minimalistic compressor module or a sort-of - just to become less or more familiar with typical dynamics processing building blocks and a variety of options they provide. Otherwise it's going to be a nightmare for you due to infinite number of unknowns.

Something like that i guess (maybe not so helpful - sorry).

__________________

Maxon

I've got excellent news: the plugin is almost finished. Unfortunately, it is not entirely true to DPLII thanks to the lack of solid documentation on the VCAs, which had been mentioned previously. I'll update this post with what I have managed to finish as of late in a few days; I want to see if there is anything I can do to bring it closer.

Maxon

What I said would take a few days ended up taking longer than a week...Oh well.
Anyway, I have one last question before I put up the code. The compression works, but Dolby Pro Logic II has a feature that allows it to change the response time in real time based on the gain at any given moment. How would I go about implementing this?

Eugene Gavrilov

I guess you need a delay line with some variable time pointer, but I suggest you send a message to Max M. concerning this

looking forward to trying your plugin

E.

Maxon

That last post got rather large! I forgot to mention that this plugin is significantly quieter than an unaltered signal. It has to be quieter to avoid distortion.

Here are some more sources to test it with:

YouTube - Mario & Sonic at the Olympic Winter Games - Official Dutch Trailer [HD]
YouTube - District 9 - Official Trailer 2 [HD]
YouTube - [JGTC] Super GT 2008 Round-Final Fuji start
YouTube - Xbox 360 - Forza Motorsport® 3 - E3 Trailer
YouTube - Forza Motorsport 3 - Aston Martin DBS Gameplay[Alpha Build] [HD]
YouTube - Gran Turismo 5 - Official E3 Trailer [HD]
YouTube - GC 2008: Killzone 2 Multiplayer Trailer 1 [High Quality]
YouTube - Gran Turismo 5 'GamesCom 09 Subaru Gameplay' TRUE-HD QUALITY

Eugene Gavrilov

Thanks for the effect! I will add it to the next driver release. Unfortunately, I cannot test it myself: I only have stereo speakers at the moment...

E.

Zorro69

Hello,

I am new here and found this thread. That´s exactly what I was looking for. Can someone explain me how to implement this Source Code in the Kx Driver. Do I have to copy it into a File an import it.

Thanks Pat

Lex Nahumury

Copy the code,
save to a text file with extension .da,
register it by doubleclicking the file,
The effect should now be listed in the DSP menu.

Zorro69

Thanks for your help, I renamed it an imported it in Kx. The first version of the plugin works fine. The newer versions I can´t import, it says wrong database. Any ideas?

Greets

Max M.

There's few misprints (missing commas before "sq" variables in register declaration section, i.e. put these missing commas and it will register).

Just in case, i guess i have to put a few remarks here (though i tried to avoid this just to not spoil the fun), to prevent a.. uhmm, let's say "misinterpretation":

Note that the code posted here (all versions) is more like a very early alpha thing - it does not really perform what's declared in the thread first post (i.e. "upmixing 2ch audio in a fashion reminiscent of Dolby Pro Logic II").

I'd suggest you first to try conventional kX's effects (installed with the driver) made for the similiar task ("stereo to surround upmixing") - for example "Surrounder(+)" or "ProLogica" - neither of two follow the "Dolby PL2" algorithms strictly, but at least they do not have major bugs/mistakes (while the code posted here does).

---
p.s. For the same reason i already recommended to Eugene to NOT include the plugin to the next driver release. Don't take me wrong but the plugin does need some ... hmm ... let's say "polishing" before.

__________________

Maxon

Yes; I got overly excited by the fact that I got anything working at all, and rushed the previous ones out before my first computer broke. This is inexcusable, especially considering that I have worked in C, so I took my time and fixed it.
To be fair (not really), I could not have located the extremely irritating popping noise or the crackling until the computer I ended up moving my code over to another computer because they did not show up on the first computer I was working on. How they failed to, I do not know, but I got rid of them. I also fixed the resource problem that Zorro69 mentioned. It occurred because I did not put semicolons after each of the "temp" lines at the beginning. It didn't dawn on me that the kxDSP editor might catch and fix those mistakes automatically. I kept it open in the background when I wasn't working on it, so it didn't get a chance to display the fixes. Regardless, I fixed it, and made sure that the plugin could be successfully uninstalled and reinstalled, as well as enable and disabled, without fatal errors.
Anyway, back to the crackling; I did not use the right constants for the multiplication during the envelope detection, so that was the problem there. The numbers were too big. The popping seems to be the result of my decision to use the external delay instead of the internal one. That part ended up being useless anyway; for what I have right now, a delay isn't even necessary. I have tested it several times to make sure of that. Any other problems were likely due to a mistake that I missed because I didn't take the time to carefully look my work over.
I seriously didn't know those were there though; had it not been for the fact that there were other people who got to hear what I had done on the first computer, I would have thought there was something wrong with my hearing when I got it working on the second.

To make this concise, I really messed up with the last ones, and I'm sorry for that. Many of the errors I should have caught, but they did not even show up when I worked on it the first time, even though they were obvious and unbelievably annoying when I had to move my work over to another computer. The one below the following part of this post is perfectly appropriate for regular listening, and even works much better than the previous editions too.

As for things that need to be dealt with now:

The compression is better; I have literally experienced no pumping at all. Since my last computer broke, I have not been able to use Dscaler 4, so I don't know how it works with it. I would imagine that there will be no problems since the horribly compressed and low quality YouTube videos play back fine.

I still need to write a minimum-phase lowpass. I'm still reading about them, and they're pretty complicated, lowpasses in general are rather difficult, so it will be a while before I get that part done.

Bass redirection is again something that I will not work on; I still have only my lousy, not full-range speakers to work with. However, correctly writing the center and front left and right channels provided bass redirection that works quite nicely. If the center channel is disabled, most of the bass disappears. Also, the center channel can be connected to both the center output and the subwoofer output for more bass, which I personally recommend. I don't necessarily recommend using the EQ Highpass filter on the surround channel outputs, as I have not checked to see if it seems necessary. From what plays back on my system, it is not. If you do use a highpass filter, set the surround one to 300/3.50. My previous motivation to use the highpass filter because all of the very low frequency sounds I hear end up in my subwoofer is no longer a problem for me while I use the plugin, by the way.

I also know for a fact that the front left and right speakers work well, but I need to refine the amplification process to improve the front-rear channel separation. While the magic of everything is not in the VCA, the VCA does tie the final package together, which I admittedly did not fully understand before. The surround channels are correctly balanced, but their behavior as an effectively separated 2 channel entity also needs work. Either way, The front and rear L and R channels are in full stereo. I have added them to halved versions of each other (see the last four lines of code) since I still need to improve the VCA elements. Unlike the buggy ones, the source does not affect the placement of stereo sounds.

Better news, even though my speakers are really bad, I did not use any external plugins while I developed this one. After I was sure I got it working the way it should have in the first place, I did use it with thedelay and the smalldelay without any issues.

I am still looking in to those other matrixing methods I mentioned before:
http://gilmore2.chem.northwestern.ed...olby1_tech.htm
I will save them for after what is here is fully finished though.

I believe that is everything that I need to mention for now, so the code is here:

I really do appreciate anyone who tries this for trusting me not to screw things up horribly again, and shortly after I have finished this post, the previous versions will be deleted.

Also, some of the stuff in that other, very long post still applies (not the bad parts, just the recommendations relating to thedelay and thesmalldelay and such). I am going to edit this one soon with the relevant parts of that post. I do remember one thing from that one right now though: if you do try this one, and something seems off, please tell me.

Maxon

I noticed the problem with using the plugin when restarting a while ago. In fact, I noticed it right after I restarted my computer to install some updates, which happened to be about an hour after that last post. However, I decided to improve on the other functions of the plugin before putting up a new version.
Again, thanks if you are still willing to try it.

Maxon

So I've come to the point where based on how my plugin has been running from the tests that I have done, it would be appropriate to release a dane source file to be downloaded. I won't, however, until the latest source code has been reviewed and seen as good enough to release.

I've rewritten, and by effect, streamlined and significantly improved the rear speaker response. There is no doubt about them being separated into a 2 channel entity at all now. The front to rear channel separation is so effective now that if I play a song that biases a sound effect to the left or right channel by any extent (Green Machine by Kyuss is an excellent example that has a 100% bias for a guitar riff in the left channel at the beginning. I would provide a link, but there are no youtube videos that preserve the stereo image) that ((X)) will use its discretion to only have the sound play in the front or the rear of the biased end of a 5.1 speaker system, but not from both speakers unless it finds such an event to be the end conclusion. To these ends, If I give ((X)) a videogame or movie to work with, it effectively steers the environment through the five speakers and everything.

Don't misunderstand me; there is still room for improvement. The front speakers are all a bit close for my liking; the separation between the left, right and center is distinct, but I'd prefer it stronger. On the other hand, physically spacing out the speakers helped quite a bit, even if I could only move them about an inch or two. The delay between the rotation of a scene and the rotation of the sound is very slight, but noticeable if extra attention is paid. I currently believe that the problem can only be solved with a minimum-phase lowpass; either that or a shorter, more efficient lowpass. However, I have left this as the last task that I will tackle in lieu of reexaminations of other sections of the code to see what can be expanded on before I begin to write a new one.

All of the previous issues that I have mentioned, except for the lowpass one, have been resolved, although I am debating on testing it with an even shorter delay (the version below uses only 6 samples).

Finally, the only similarities between what I have finished and what I had posted above is that they share the same foundation and the latest one grew out of them. In other words, this version is, for all intents and purposes, distinct from them outside of the lowpass and variable names. I almost forgot, several variables have been removed too.

Oh yeah, watch the PGR 4 Tokyo video. This new version makes it awesome!

BuzzT

@Maxon: Your plugin is amazing. I just tested it with my 5.1 Speakers (Teufel E Magnum Power Edition).
Somehow the registration of it was a bit tricky. An error occured called "wrong plugin library", but I got it working by using a semicolon after "itramsize 6".
Your latest release is really a big step forward. Thanks for your efforts.

Maxon

I always manage to miss a semicolon...:/
Sorry about that. I appreciate your response, and I was wondering if there are any specific problems that you have run into so far.

Maxon

I have substantially improved the stereo width of the plugin. I have also successfully added in variable delay lines for the left-right and front-rear separation. I put the front-rear on the itram and the left-right on the xtram because they were much easier to distinguish that way. The only thing I have left to do now is write a minimum-phase lowpass.

peate

I gave your plug-in a try on my old Live 5.1 and it rocks. Good job.It left room for more EQ's and effects.

Thanks.

BuzzT

Hello Maxon, any news about your "((X)) Surround System"? It's in use almost one year on my surround speaker setup. Like peate said - it really rocks!