Refining the DDS approach

This is another guest post by André, after revising and improving his (to my knowledge) completely new and untried DDS technique, he is achieving splendid results.

Hi Batperson and fellow DIY’ers.

I have focused on two major areas in my efforts to improve the colour osd prototype.

Fixing the chroma inconsistency

The first task was to deal with the perceived chroma inconsistency, which manifests as colour variations rolling vertically up or down. After much back and forth with some ideas and coding, I found that a partial answer was lying just in front of me. However the issue still needs to be tackled on more than one front to be fully resolved. I realized that there were two major contributors:

MCU comparator timing variations
DDS square wave output.

The trigger point of the comparator varies due to factors such as power supply noise or video signal voltage variations, and this inconsistency of the comparator is responsible for some colour variation. However the DSPIC on the prototype has a comparator filter, which when enabled gives a more consistent trigger point at the expense of a reduced sampling rate. After much fiddling with parameters I settled on a 1:5 prescaler, as the comparator is sampled at 400Mhz, the filter samples at 80Mhz, and it takes 3 consecutive positive samples to produce a change in the comparator output. So in reality the filter has a sampling output of 26.666Mhz (curiously, or not, this frequency is suspiciously close to the 27Mhz sampling rate of modern video decoders). Even after many sessions adjusting the filter sampling rate the issue never disappeared and I only obtained small improvements. However, that lead me to another solution, this time more accidental.

The first time I tried to connect the DDS output to the voltage created by the mcu pin, the DDS was outputting its frequency signal as a sine wave. Unfortunately it didn’t have any effect, and I was convinced that the DDS DAC output was too weak to make any change to the voltage already set by the port pin 330Ω resistor to the 75Ω input load voltage divider as seen in my previous post.

The DDS DAC only outputs a maximum of around 0.680V for sine or triangular waves, depending on frequency. Because of that I changed to square wave output which is a 0V – 3.3V signal and in my excitement at finally getting a colour overlay I never looked back at this fact.

As I could not fix the Chroma inconsistency issue, I reflected on the causes of the abrupt variations of color, and I thought for a while that the quadrature decoder inside the video receiver (monitor or video digitizer) would not like to receive square waves, but that idea was not convicing me, as I tried to improve the chroma detection and the DDS resetting routine.

One day out of tiredness and hopelessness, I reverted the DDS to triangle wave output, and the improvement was better than I imagined.

Improved colour bars — Super smooth colours!

Keep in mind two things: the video digitizer I used has a resolution of only 640×480 but the overall resolution looks even lower, and the DDS output is too weak to make a true 1V p-p sinewave so the digitizer’s automatic gain control boosts the signal making the image appear brighter. I will amplify the signal to proper levels later on.

This solution also made me modify the connection between the DDS output and the resistor divider output:

The 100pf capacitor is a simple AC coupling capacitor, nothing fancy.

Also there are signs of impedance mismatch as the overlay is switched on and off from the video signal, visible at the start of the overlay as a brightness change, and also at the end of the overlay as well if you look closely. As an interim solution I added a 75Ω resistor connected via a diode to the pixel signal pin, so that when the overlay is switched off the port pin that controls the switch is terminating the signal, when it is on the diode prevents the pin’s 3.3V from entering the video signal path and making all the overlay white, and potentially damaging the receiver input stage or camera output stage.

After I acquired a better video digitizer and changing the DDS to sine wave output the image became even better, IMHO.

Further improved colour bars

As previously mentioned, brightness and color saturation were increased in the PC software due to the weak DDS DAC output. I intend to address this later.

The artifacts at the numbers are caused by the way the digitizer deals with interlacing, these numbers vary at double the frame rate of NTSC (I just now realized this so I will correct it ASAP). Curiously the old digitizer dealt better with interlaced signals, maybe the crappy resolution masked this.

The vertical color rolling effect is barely perceptible now, and the quality o the overlay is vastly improved.

So I conclude that if you are trying to make a color OSD, you want to generate your overlay using sine or triangle wave signals, as the abrupt transitions of a square wave lead to colour inconsistencies. This happens because square waves are composed of various odd frequencies besides the fundamental frequency and this will certainly end up confusing the receiver converter.

Improving the firmware

The second area I wished to improve was the synchronization with the incoming video as I needed a stable overlay to continue the improvement of the OSD code without constant overlay dropouts.

This took up the vast majority of the coding efforts of the past weeks, and even if it is not fully fixed it has improved by leaps and bounds.

The problem would manifest itself as a complete disappearance of the overlay in case of sharp brightness variations from one scanline to another. One of the reasons of this issue is the attempt by me to let go of the “more silicon syndrome” by avoiding the use of popular sync stripper chips such as the famous LM1881. However replacing a venerable chip like the LM1881 proved to be a gargantuan task, as it is an old but still effective chip, with lots of modern descendants.

The prototype uses an internal MCU comparator to achieve many of the tasks of a sync stripper, adjusting an internal DAC connected to the comparator. From the comparator signal a routine was created to firstly increase the DAC voltage until the sync signals are detected, and then to adjust the DAC signal until it is just slightly below sync level (the voltage just below Black level) and then hold this voltage.

The structure of interrupts of the code were as as seen in the next figure.

One disavantage of the “Comparator does all” approach, is that the composite video signal is a complex signal, having slow wave forms like the sync tip, and fast modulated signals, like the chroma and luma, and also variations of the image brightness will introduce variations on the sync level voltage, even with a clamping circuit. These variations are called APL, average picture level. This factor led me to introduce a “dynamic DAC voltage detection”. Simply put the MCU would use an ADC to read the sync tip voltage at interrupt 1.

The current DSPIC used in this prototype has a very useful ADC peripheral that besides being a 3.5Msps ADC, also has an oversampling or averaging filter.

Oversampling is not useful as the ADC resolution is a generous 12 bits, and it coincides with the 12 bits of the comparator DAC. Instead the ADC filter was used at 8X averaging. In this mode, the ADC when requested will automatically take 8 measurements and average them before presenting the value in a register. Using that value, the program extrapolates the sync level voltage, and ideally keeps it from going over the sync level threshold.

The solution proved to be insufficient. If the next line had a big brightness change like the sun appearing at full brightness, especially at the right edge of the image, right before the next sync tip, the APL would change up and interrupt 2 would be lost, and because it missed the 4 chroma burst raising edges, the interrupt would never fire. Consequently the overlay would disappear because this interrupt decides whether the line is a scanline or vertical sync line.

An interrupt rearrangement needed to be devised to give sufficient time, meaning more cycles, to implement the DAC dynamic adjustment before the chroma burst, as seen in the next image.

For this, interrupt 1 was modified to wait for the ADC filter result, decide if the sync tip was a new Scanline or a Vertical sync line and raise the DAC voltage for the chroma detection. And interrupt 2 was changed from CCP reading 4 raising edges (it has to be under 8 ideally), to a DMA interrupt that takes 6 or 7 (still deciding on this) chroma rising edges from the same CCP, giving more time between these two interrupts. It then resets the DDS, and lowers the DAC voltage to the previous sync detection level to avoid any scanline glitches causing an interrupt to fire in error.

Even at 8X averaging the ADC voltage readings would oscillate plus or minus 2 lsb’s, so to improve that, and also deal with extraneous noise induced by the MCU or video signal, a median filter was applied afterwards in the line rendering interrupt (call it phase 3) to filter out noise outliers and implemented before the next sync detection.

A median filter has the advantage of filtering out large measuring spikes by sorting a list of past values and then taking the value at the middle of the list. The list size is currently 23 values.

The interrupt structure change decouples line detection from the chroma burst detection, meaning that in case the chroma burst is missing, such as with a black and white composite video image, the overlay will still be rendered, although with unsynchronized colours.

The results can be seen at these videos. The brightness of the video changes dramatically, but the MCU remains locked to the incoming video sync.

Note: these videos were made with different cameras, which shows another benefit of the dynamic DAC ajustment: better camera compatibility.

Next up will be a general code cleanup, increasing the amplitude of the DDS-generated chroma, implementing a luma DAC using resistors, and finally a long-promised full schematic. There are still issues with the overlay ocasionally disappearing, but this only occurs now under extreme conditions.

Thanks to my wife, family and friends, including the nice host of this blog, Batperson, for letting me publish my efforts and ramblings.

Milstan (André)

A new approach using DDS

This is a guest post by André who has come up with a novel approach using Direct Digital Synthesis of the chroma signal. The circuit is relatively simple and uses few components. It’s early days, but he’s already achieved promising results. Take a look at how he does it!

DDS prototype

This is a report of my attempt at creating a simple, low component overlay system with colour, for use in remote controlled vehicles, in an attempt to change the current go to hardware for doing video overlay, namely the character only monochrome Max OSD chip.

After much research and discussion with fellow Batperson from the WekaOSD blog I set out to devise a solution for colour overlay that uses the smallest number of components possible.

In the first part of the project I set out to create a monochrome OSD using the DSPIC33CK, as it is the MCU that I’m most comfortable with, despite the buggy IDE, because of the sheer number of peripherals it has: comparators with DAC references, timers, capture and compare modules, ADC@3.5MSPS. That way I could integrate as many analog blocks as possible, avoiding noise issues and lowering part count. It is also a 100MIPS MCU, the majority of the instructions are single cycle, and interrupt latency is fixed if you obey some restrictions.

For sync detection I used an internal comparator fed by the video signal, where the reference voltage was controlled by the comparator dedicated DAC, to separate any unwanted signals the comparator input would present on the video feed. I used a solution inspired by the schematic of Dennis Frie’s DIY OSD. The diode is a standard 1N4148.

clamp_comparator

For rendering I chose an analog switch, namely the SN74LVC1G3157 SOT-23 (SPDT) single channel with typical frequency of 340 MHz, connected in such a way that in case of MCU malfunction the video feed would pass on via the normally closed input. Rendering is made by an MCU pin on the normally open input. This input receives 3.3 volts via a 330 ohms resistor to create a signal above 0.3v which gave me a nice dark grey overlay. (Voltage divider with the 75 ohm termination resistor inside a monitor).

Then came the daunting task of generating colour, and here I made some bold assumptions. Simply put, my idea was to generate a 3.579545 MHz waveform using a frequency generator, and try to synchronize this to the detected chroma burst in an interrupt. In this way I would have a chroma signal in sync with the video feed, circumventing the need for phase-frequency lock using obsolete parts or a multiple chip solution. So I set out to find a frequency generator chip that was both small and required only clock input from the MCU and some type of control SPI or I2C.

After some search on the web I decided to use a DDS chip, and following the constraints I imposed I settled on the AD9833 chip in TSSOP 10 pin format. To simplify prototyping I opted to buy a PCB module of the AD9833 and simply removed the 25Mhz crystal and fed it with a frequency generated by the MCU, so the DDS was configured to output the Chroma frequency, in square wave format to get the benefits of the 0 – 3.3v amplitude of the DDS output.

ad9833

One particularity of this chip is that the best phase noise is achieved with a frequency output that is a whole divisor of the input clock, as in the first images I got the phase inconsistency was visible.

The solution was an undocumented register of the DSPIC – REFOTRIML, that allowed me to trim the frequency to about 7 times the chroma frequency, 25.056818Mhz. That solved the phase inconsistencies by using a multiple of the input clock of the DDS. That is also a slight overclock of the DDS (max 25Mhz).

horz_scan_phase_inconsistency_2 — DDS phase consistency

The next step required me to mix the dark grey luma signal created by the resistor divider with the generated clock signal, so I used the technique often found on resistor DACs, feeding the clock output via another resistor, effectively creating a crude DAC.

mixing_luma_chroma

In my code I opted to synchronize with the incoming video signal in 3 phases:

first phase was to detect the video sync tip falling edge. That is accomplished with the comparator interrupt. This would provide a reference for the start of the overlay signal, by enabling a timer interrupt that would fire in the visible part of the scanline, and at the same time the code enables an input compare interrupt to detect the chroma burst cycles.

Second phase, the input compare interrupt fires on 4 rising edges. The first of these is the sync rising edge and the next 3 are the rising edges of the chroma burst. Right at the start of the interrupt the code resets and restarts the DDS at a defined phase, ideally one that would correct any frequency drift due to mismatched frequencies between the video effective chroma frequency and the local generated frequency in each scanline. Next, by reading the first buffered rising edge (the rising edge of sync tip) it will determine if the signal is a scanline, short sync or long sync, and keep track of the current scanline number. Each of the options performs different actions. If it is in a scanline it hands off to the rendering interrupt, otherwise it resets the line counter and disables the render timer and render interrupt.

Third phase, the rendering timer interrupt, basically renders according to the line number. To transition to the different colours in the colour bars, it pauses the clock feeding the DDS for 10ns (ideally; in actuality it takes more like 20ns). That equates to 14 shades of colour, theoretically it should be 28. At the end of this phase before returning from the interrupt, the timer and timer interrupt are disabled, and the comparator interrupt is re-enabled.

comparator_output

At present I still think it is still a complicated scheme of interrupts that needs further simplification. However although the results leave room for improvement, they are very encouraging.

NTSC Colour bars — 14 hues of colour are being generated, overlaid onto a video signal. Some phase noise is present in the form of rolling vertical colour changes. However colour is consistent horizontally across each scan line.

Each column represents a 10ns delay in code, but in reality enabling and disabling the clock input of the DDS takes 2 cycles as it is visible that the colour cycles in approximately 14 cycles (each chroma cycle takes ~28 cycles) and there are contiguous columns that are the same colour despite the phase shift. As can be seen, there is phase shift vertically in each column, in the form of a colour shift rolling up along the columns.

Further investigation on jitter leads me to suspect the random nature of the colour shifts, but further investigation will be carried out, I have not scoped the signal completely. In the next figure, for each column the DDS input clock was stopped and a line rendered, depending on whether the output DDS clock was on a high level (white) or a low level (black). In theory the columns should alternate between white and black, and the lines should also alternate regularly between white and black due to the 180 degree phase shift between lines (in NTSC, each line should contain exactly 227.5 chroma subcarrier cycles). However as can be seen this is not the case. I think that because of the software-only line rendering routine some jitter may come from IF…THEN, and other C code instructions. Even utilizing some ASM did not mitigate the jitter.

jitter_investigation

In the next weeks or months, if time permits I will restructure the code and try to solve other issues, and attempt to minimize the rendering interrupt jitter using DMA instead of bit-banging. I still haven’t ruled out EMI on the comparator input as a source of jitter, but the comparator has a Software Hysteresis setting of none, 15, 30 or 45 mV and a digital filter with prescaler. Adjusting both did not improve the jitter.

Some solutions I thought about is putting an XOR gate on the DDS input frequency line to do proper 10ns delays, and supporting greyscale by putting another XOR gate between the DDS and the resistor DAC. Another thought is to clock the MCU at 3x27Mhz, as modern video monitors sample the video signal at 27Mhz.

Final thoughts: The current approach of generating colour shifts by delaying the DDS input clock 10ns (in practice 20ns) is feasible as a proof of concept but it would require code to keep track of the delays if arbitrary colours are to be generated. However I want a complete rendering interrupt that will simply fire the DMA or DMA’s and return to main, so that other tasks can be performed at the same time like reading a UART and creating a rendering buffer, perhaps a double buffer.

My regards.
André
Aka “Milstan”

Thanks to Batperson for the ideas shared, that gave me inspiration to create this prototype.
Thanks for Sónia for all the support and putting up with my nerdism.
Thanks to my family for everything.