A True Bhartiya Swami Ramdev ji speaking in Hindi at ITConclave 2012 India

Dear Friends,

I am feeling from sometime that I am watching so many good, informative and entertainment videos which may helpful for others too. But keeping it upto me only, does not seems fair to me and for others ....

So I decided to open video blog and put all the videos to watch by my readers. But it needed a new blog, new webpage and bla bla. In spite of starting new blog I choose to put new lablel 'video blog' in my current blog and continue posting in the current blog only. By doing this way, you dont have to switch to different blogs and all the information will be available here in one stop.

I hope you are enjoying my blog by reading the posts and watching the videos.

A True Bhartiya (Swami Ramdev ji) speaking in Hindi at #ITConclave in front of so called Creamy layer India.  He has answered in very polite manner all the questions rudely asked by Chetan Bhagat. Good one to listen him.

Happy Thinking Naturally..........

Back To Basics : YUV to RGB conversion (Color Space)

I know there are like hundreds of books and tutorials available which explain color space. But I also know that neither of those tutorials stop a new author to explain it again in different manner, more explanatory in more easy way, so nor Me.

I remember my first interview in video domain. I can still hear that voice.
"Ok tell me what will be the color if Y 128 and Cb and Cr are 0", I said "Gray"
"if Y is 0 and Cb and Cr are at 128 then color is ", I was not sure what to say....
"Ok all are in 128, so what will be color", I said "White" (It was my guess)

I must say that interviewer taught me a lot, At that time I was mostly working on understanding of H.264 standard. That was a hell work for me. So I was learning how to implement those algorithms but for basics of video thing I was just beginner.

ok, lets start the tutorial now.......

Color models are conversion formats from one model to another. There are RGB, YCbCr, YUV etc. RGB in formations comes though the capturing devices like cameras or scanners. RGB color space is generally used in computer graphics. And combination of these three colors will generate other colors like white, black, yellow, cyan etc.But in real world processing RGB model is not the efficient one. If each color represents with 8 bits/pixel than for RGB we need 24 bits/pixels. If we wish to modify intensity or color of one pixel, we have to read all three components, process it and then store back, NOT good in real time with processing wise as well as memory requirement wise. To solve that problem there are other color space/model which store the pixel information in intensity and color format, like using luma and two color difference which can be converted to RGB or from RGB to that formats. Most common is YUV format.

The basic equations for YUV are :
Y = 0.299R + 0.587G + 0.114B
U = 0.492 (B-Y)
U = 0.887 (R-Y)

and YUV to RGB are :
R = Y + 1.14V
G = Y - 0.395U - 0.581V
B = Y + 2.032U


RGB ranges 0 to 255 while Y has a range of 0 to 255, U ranges 0 to +/- 112 and V ranges 0 to +/-157. But these equations are generally scaled for better implementation in NTSC or PAL digital codec. And for 8 bit data YUV and RGB data are saturated between 0 to 255.

YCbCr is scaled and offset version of YUV model, where Y ranges 16-235, Cb and Cr ranges 16-240. But actually all saturated to 0-255 levels. Here are the equations:

Y = 0.257R + 0.504G + 0.098B + 16
Cb = -0.148R - 0.291G + 0.439B + 128
Cr = 0.439R - 0.368G - 0.071B +128

and YCbCr to RGB conversion equations are :
R = 1.164 (Y-16) + 1.596 (Cr-128)
G = 1.164 (Y-16) - 0.813 (Cr-128) - 0.391 (Cb - 128)
B = 1.164 (Y-16) + 2.018 (Cb- 128)

As per my interview Q & A I have to consider here to this YCbCr to RGB conversion. If you analyze properly YUV to RGB and YCbCR to RGB equations are almost the same. I simplified the equation for better to remember way for human being like me and specially should not be used in computer for implementation. (computer dont give interview ;))

Here is my approximate way for YCbCr to RGB conversion:

R = Y+ 1.5 (Cr-128)
G = Y - 0.8(Cr-128)
B = Y+ 2.0 (Cb-128)

Here is the table for color we will get with different value of YCbCr.

You dont have remember this table, just use above approximate equation and RGB Color Cube (shown below), you will have the answer in sec.



By the way, after these tutorial, here are my answers for previous questions.
1) what will be the color if Y 128 and Cb and Cr are 0 ? GREEN
2) If Y is 0 and Cb and Cr are at 128, what will be the color ? BLACK
3) All are in 128, so color will be ? WHITE

And this time 100% sure... :)

Clip Signed Data To Arbitrary Unsigned Range in SIMD/Assembly

This post is again dedicated to Video domain. But I certainly can say there are various other applications too where we can use it.

Clipping is very simple algorithm as it name indicates, we clip our data to certain range. There will be High value and Low value. If data value is less than Low value, assign data to Low and if data value goes upper than High value assign data to High. something like this:

Clip(Data,Low,High) = if Data is less than Low then data = Low, else if Data is greater than High then data= High , else data = Data ------------(1)

In video domain, after IDCT operation we get signed data for pixel, which should be technically unsigned data type. Here we do clip operation for pixel data, and limit the data between unsigned data type range, and in normal situation pixel bit depth is 8 (i.e unsigned char). So equ. (1) becomes :


Clip(Data,0,255) =if Data is less than 0 then data = 0, else if Data is greater than 255 data= 255, else data = Data ------------------------(2)


But pixel bit-depth is not limited to 8. As I mentioned in my previous post "H264:How to do conversion from 8 bits to 14 bits bit depth support" under the label "Video", H.264 support till 14 bit bit-depth, when you don't want to compromise with quality, go for higher bit -depth. And here pixel data type will be 'unsigned short'. Now we have to modify the 'Clip' function for higher bit-depth. And this time it is not fixed to 8 and not even 14, rather it can vary from 8 to 14 depends upon the YUV input bit-depth for encoder and luma or chroma bit-depth information (bit_depth_luma_minus8 and bit_depth_chroma_minus8) from input H.264 coded video input for decoder. So lets do this clipping in generic form. And remember Low value will be 0 always only High value will change. So equ (2) modified as :

High = (1^Pixel_Bit_Depth)-1
Clip(Data, 0, High) = if Data is less than 0 then data = 0, else if Data is greater than High then data= High , else data = Data ------------------------(3)

There are other optimized ways too for equ.(3), but that's not my concern as of today. so moving ahead for SIMD/assembly (MMX/SSE/SSE2). Now how to achieve the same operation in assembly. Actually if bit-depth is 8 then there is a single instruction available in SIMD as :

packuswb Rx0, Rx0 ;Considered data is in Rx0 (mm/xmm) SIMD register (if pixel type is unsigned char)

or if Rx1 is '0' then

paddusb Rx0, Rx1

if you want to saturate for unsigned short then we have

paddusw Rx0, Rx1

But that is not our case, so we have to go by other way. As we have data type 'unsigned short ' and Max value will be (1^Pixel_Bit_Depth)-1 , So here we goes :

unsigned short High = (1^Pixel_Bit_Depth)-1
unsigned short Range = 0x8000;
unsigned short Low = 0x7FFF - High;
unsigned short MaxHigh = 0xFFFF - High;

movdqu Rx1, Range
movdqu Rx2, Low
movdqu Rx3, MaxHigh
paddw Rx0, Rx1
paddusw Rx0, Rx2 ;Add unsigned saturation with Low
psubusw Rx0, Rx3 ;Subtract unsigned saturation with MaxHigh


(Note above code instructions are for SSE2 but applied for MMX too, also I tried to wrote for one pixel data, to use SIMD advantage properly some data shuffling is required, here my intention was to give idea, not the complete code for cut n paste.)

Enjoy!!!!

From Unreality Magzine...

I was just stumbling through various sites and got this article. And felt that I should shares with you. The article is named as :

The 10 Most Visually Stunning Movies of the Last 10 Years

According to Publisher these are the movies which change the way we view movies. These movies are visually unforgettable with their heavily loaded graphics magics.

And from video codec point of view also, the movies listed here are great test vectors for video compression encoder tools. Specially like 300, The Matrix Reloaded, Transformer.


Just check this article and see the movie list .... with some movie snaps...
If you didn't watch those...update your 'must watching movie' list.. ;)

And on this list ... I want to add some more movies like
1) The Lord Of The Rings
2) The Fountain (it is in my 'must watching movie' list, I saw it's trailer and that's awesome )

What's your views say........
Want to add some more movies ...

SSE2 Vs SSSE3

As you already know that I am a bit busy with learning intel SIMD like mmx, sse2 , ssse3 etc stuff. I am enjoying SIMD and playing with all these MMX, SSE versions.While working with SSSE3 after sse2 or sse3, I thought what is the advantage of SSSE3 over SSE2? Some people even ask me why there is not a dramatic change in performance after adding SSSE3. I knew the answer but thought to do some more R&D on it.

And here is my view....

I will start with some brief intro of SSE versions and also as I am in video field I will talk about integer operations only that will be my primary concern as of now.

SSE2 instructions are an extension of the SIMD introduced with the MMX technology and the SSE extensions.The key benefits of SSE2 are that both MMX ans SSE2 instructions can work on 8 XMM (128-bit, XMM0- XMM7) register along with the MMX registers (mm0-mm7), and that SSE instructions now support 64-bit floating-point values. So there was huge change between MMX and SSE2(or SSE). Now because of XMM registers instead of playing with 8 bytes, we can play 16 bytes simultaneously. So improving the performance just by double from the MMX assembly or 16 times from the C code. There are some instructions we are missing in MMX assembly which are present in SSE2 like paddsb/w, movapd,movupd, pshufw/d ,pavgb/w etc, which are very much helpful here in video compression.

While SSSE3 (Supplemental Streaming SIMD Extension 3) is an extension of SSE3 or I should say revision of SSE3. In SSE3(13 new instructions) the most notable change is the capability to work horizontally in a register, as opposed to the more or less strictly vertical operation of all previous SSE instructions. There are instructions to add and subtract the multiple values stored within a single register have been added. But note those are not for Integer operations only floating point, that's why I am talking about SSSE3.

SSSE3 contains 16 new discrete instructions over SSE3. Each can act on 64-bit MMX or 128-bit XMM registers. Therefore, Intel's manuals has 32 new instructions.The instructions are PSIGNB/W/D, PABSB/W/D, PALIGNR, PSHUFB, PMULHRSW, PMADDUBSW, PHSUBW/D, PHSUBSW, PHADDW/D and PHADDSW.So if you these, the processing block or registers are same as SSE2, no new registers.

By using SSSE3 the only advantage in video compression side integer operations is horizontally processing. So by SSSE3 we can add/subtract the data within the registers instead of adding or subtracting with other registers. So I feel SSSE3 only removes some overheads and save some cycles by using horizontal operations if your video code is having that kind of module like SAD, SSD and all, but there are be many places where transition from MMX to SSE2 gives huge improvement in performance but transition from SSE2 to SSSE3 may not give you even noticeable change. Even there will be lots of functions where SSSE3 will not be required over SSE2 in code. As to work vertically (between two registers) we sometimes do some data manipulations by padding 0's or by shuffling data between registers, and then process the data like addition/multiplication etc., those shuffling or padding are overheads that can be avoided here in SSSE3.

I guess we should not think that each next generation of SIMD will just magically double the performance of the code same like MMX to SSE/SSE2. Function module (like DCT, SAD etc.) and data fetching to those functions matters a lot to decide which SIMD we should use .... SSE2 or SSSE3 for better performance. So before converting any new code from SSE2 to SSSE3, just stop for a moment, have a close look on the module and then choose SSE2 Vs SSSE3.

Enjoy SIMD optimization.

NGVC (H.265) Is On The way

While the whole multimedia world trying very hard to become mature in H.264, the bestcompression video standard till today, the new baby is in under development phase and named as 'The Next Generation Video Coding' (NCVG). In 2005 it was started by VCEG as consideration to 'H.264+'. Then after study it changed to H.265 a brand new standard instead of an extension of H.264 as a long-term video coding standard. And now latest VCEG meeting it has came up as 'NGVC' project (next-generation video coding) with backward compatibility. It is expected to be finalized in 2009-2010.

The goal of this standardization will be as follows:

1. Coding efficiency:

* NGVC should be capable of providing a bit rate reduction of 50% at the same subjective quality

2. Complexity:

* NGVC should be capable of operating with a complexity ranging from 50% to 3 times H.264/MPEG-4 AVC High Profile.

* When operated at a complexity of 50% compared to H.264/MPEG-4 AVC High Profile, NGVC should provide a 25% bit rate savings compared to H.264/MPEG-4 AVC High Profile at equivalent subjective quality.

3. Applications:

* Low-delay interactive video communications
* Surveillance
* Streaming
* Broadcast
* Digital cinema and large-screen digital imagery
* Mobile video entertainment
* Storage-based video application (camcorders, camera phones, computer files, disc media, download-and-play, etc)


KTA (key technical area) is developed as the software platform, which uses JM11 as the baseline and continuously integrates promising coding tools. The tools adopted in KTA are listed as below:

* 2-D non-separable adaptive interpolation filter (AIF) [AD08]
* separable AIF [C-0219-E]
* directional AIF [AG21]
* motion compensation with 1/8-pel motion vectors [AD09]
* adaptive prediction error coding (APEC) in spatial and frequency domain [AD07]
* adaptive quantization matrix selection (AQMS) [AD06]
* competition-based scheme for motion vector selection and coding [AC06]
* mode-dependent transform customization for intra coding [AG11]

All these techniques improve the coding performance by multi-pass encoding.

The latest published KTA software is JM11.0KTA2.3 (download here ). Some new technologies have been adopted by KTA software since July 2008. Those KTA coding tools involve the following areas:

1. Architecture

Internal Bit Depth Increasing
Extended Block Size (or called Super-MacroBlock) (C123)

2. Transformation and Quantization

Mode-Dependent Directional Transform
Very Large Block Transform
Adaptive Prediction Error Coding
Improved Adaptive Quantization Matrix Selection
Rate-Distortion-Optimization Quantization (RDO-Q)
Adaptive QP

3. Entropy Coding

Parallel Entropy Coding

4. Adaptive Loop Filter

Block/Quadtree-based Adaptive Loop Filter (C181)

5. Motion Coding

Motion Vector Prediction Competition
One-eighth-sample Motion Vector Resolution

6. Inter-Prediction

Adaptive Interpolation Filters
Separable Adaptive Interpolation Filters
Directional Adaptive Interpolation Filters
Enhanced Adaptive Interpolation Filter
Enhanced Directional Adaptive Interpolation Filter
Fixed Directional Interpolation Filters
Special Filter Positions
High Precision Filters
Switched Interpolation Filters with Offsets

These are some related helpful links:

1) ITU-T SG16’s homepage
2) The latest version of KTA is JM11.0KTA2.3 (Download here) and the latest test conditions are specified in [AH10].
3) H265.net forum instead use this H.265.net (modifying after a valuable comment...thanks for rectifying me)


So be ready to see new fun in video coding.

H264:How to do conversion from 8 bits to 14 bit support

There are lots of professional applications which require higher bit depth support like studio application, HD application. In H.264 out of 11 profiles there are 7 profiles which supports more than 8 bits bit depth starting from High10 which supports 10 bits bit depth. There are High 444 Predictive and some related profiles which support upto 14 bits. Anyway the conversion procedure wise both are pretty much same except the specific values.

One more things we should keep in mind that bit depth may be different for Luma and Chroma components(both Cb and Cr will be of same bit depth).

So here I am describing the process conversion of encoder/decoder for than 8 bits, lets say specific to 14 bits support. For simplification I am taking both Luma and Chomra compo nets are of equal bit depth of BitDepth =14.So for this case BitDepthY = BitDepthC = BitDepth.

Note:For standardization reason, before that you must support at least main profile.I will put corresponding equation with equation number from the standard version ITU-T Rec. H.264 (11/2007) .

1)Generally for pixel variables we use 'char', first thing is convert this to 'short'

2)Change all your variables related to pixel/samples for 'short' like arrays, pointers, file read , file write , memcpy etc.

3)Change your 'clip' functions for pixels according bit depth for both Luma and Chroma components.
Clip1Y( x ) = Clip3( 0, ( 1 << BitDepthY ) – 1, x ) (5-3)
Clip1C( x ) = Clip3( 0, ( 1 << BitDepthC ) – 1, x ) (5-4)

4)Now decoder has to know the bit depth of the pixels so it has to read 'bit_depth_luma_minus8 ' and 'bit_depth_chroma_minus8 ' in the SPS header. With these parameters find out 'BitDepthY ' and 'QpBdOffsetY ' and similarly for chroma components.
BitDepthY = 8 + bit_depth_luma_minus8 (7-2)
QpBdOffsetY = 6 * bit_depth_luma_minus8 (7-3)

And
BitDepthC = 8 + bit_depth_chroma_minus8 (7-4)
QpBdOffsetC = 6 * ( bit_depth_chroma_minus8 + residual_colour_transform_flag ) (7-5)

In the encoder side the 'bit_depth_luma_minus8 ' and 'bit_depth_chroma_minus8 ' should be send in the SPS header to .264 bitstream.

5)As now each sample has bit depth of BitDepthY for luma and BitDepthC for chroma components , the PCM samples of I_PCM should be accordingly modified.

6)For intra prediction DC prediction mode value will change according to BitDepth.
pred4x4L[ x, y ] = ( 1 << ( BitDepthY – 1 ) ) (8-52)
pred8x8L[ x, y ] = ( 1 << ( BitDepthY – 1 ) ) (8-96)
predL[ x, y ] = ( 1 << ( BitDepthY – 1 ) ), with x, y = 0..15 (8-123)

And as well as Chroma components
predC[ x + xO, y + yO ] = ( 1 << ( BitDepthC – 1 ) ), with x, y = 0..3. (8-139)
predC[ x + xO, y + yO ] = ( 1 << ( BitDepthC – 1 ) ), with x, y = 0..3. (8-142)
predC[ x + xO, y + yO ] = ( 1 << ( BitDepthC – 1 ) ), with x, y = 0..3. (8-145)

7)If you are using prediction weights then some work we have to do here also.
o0C = luma_offset_l0[ refIdxL0WP ] * ( 1 << ( BitDepthY – 8 ) ) (8-295)
o1C = luma_offset_l1[ refIdxL1WP ] * ( 1 << ( BitDepthY – 8 ) ) (8-296)

And for chroma components
o0C = chroma_offset_l0[ refIdxL0WP ][ iCbCr ] * ( 1 << ( BitDepthC – 8 ) ) (8-300)
o1C = chroma_offset_l1[ refIdxL1WP ][ iCbCr ] * ( 1 << ( BitDepthC – 8 ) ) (8-301)

8)As bit depth of pixels has changed so it will affect a lot to quantization.

1.'pic_init_qp_minus26' range will be now -(26 + QpBdOffsetY ) to +25, inclusive.

2.SliceQPY will be in the range of -QpBdOffsetY to +51, inclusive.
SliceQPY = 26 + pic_init_qp_minus26 + slice_qp_delta (7-28)

So if we have bit depth of 14 so our SliceQPY will be in the range of -36 to +51.

3.'mb_qp_delta' will be in the range of –( 26 + QpBdOffsetY / 2) to +( 25 + QpBdOffsetY / 2 )
The value of QPY is derived as
QPY = ( ( QPY,PREV + mb_qp_delta + 52 + 2 * QpBdOffsetY ) % ( 52 + QpBdOffsetY ) ) - QpBdOffsetY (7-35)

And the working QP for luma components will be QP'Y , which is derived as
QP'Y = QPY + QpBdOffsetY (7-36)

Remember QP quantisation parameter values QPY is always in the range of –QpBdOffsetY to 51, inclusive. QP quantisation parameter values QPC is always in the range of –QpBdOffsetC to 51, inclusive.

4.For the chroma quantization parameters the value of QPC is determined from the current value of QPY (NOT QP'Y)and the value of 'chroma_qp_index_offset' (for Cb) or 'second_chroma_qp_index_offset' (for Cr).

If the chroma component is the Cb component, qPOffset is
qPOffset = chroma_qp_index_offset (8-315)

Otherwise (the chroma component is the Cr component), qPOffset is
qPOffset = second_chroma_qp_index_offset (8-316)

The value of qPI for each chroma component is derived as
qPI = Clip3( –QpBdOffsetC, 51, QPY + qPOffset ) (8-317)

And QPC = Chroma Quantization table[qPI]

Finally
The value of QP'C for the chroma components will be
QP'C = QPC + QpBdOffsetC (8-318)

5.The variable qP for quantization wil be QP'Y for luma components and QP'C for chorma components.

9)The bit depth also affect in deblocking process.

1.For average quantization parameter qPav the qPp and qPq will be correspond to QPY for chromaEdgeFlag equal to 0 (luma components) and QPC for chromaEdgeFlag equal to 1 (choma components).

2.Threshold variables a and ß will vary as
If chromaEdgeFlag is equal to 0,
a = a' * (1 << ( BitDepthY – 8 ) ) (8-466)
ß = ß' * (1 << ( BitDepthY – 8 ) ) (8-467)

Otherwise (chromaEdgeFlag is equal to 1),
a = a' * (1 << ( BitDepthC – 8 ) ) (8-468)
ß = ß' * (1 << ( BitDepthC – 8 ) ) (8-469)

3.Threshold variable tC0 will vary as
If chromaEdgeFlag is equal to 0,
tC0 = t'C0 * (1 << ( BitDepthY – 8 ) ) (8-476)

Otherwise (chromaEdgeFlag is equal to 1),
tC0 = t'C0 * (1 << ( BitDepthC – 8 ) ) (8-477)

So now we are ready for professional applications with 14 bits bit depth support for higher quality and by providing best compression with the power of H.264.

Tip for the topic: As you changed all pixel related data types to 'short' to support mote than 8 bits bit depth, just check for input which has 8 bits bit depth only. Is your code working fine???

I guess you dont want two different code base for 8 bits and more than 8 bits.Think hard and think naturally...you definitely dont need two different code base... ;)