VIP Video Converter Manuel d'utilisateur

© 2012 Altera Corporation—Confidential Altera Video System Building / implementing video systems

© 2012 Altera Corporation—Confidential Qsys-Supported Standard Interfaces  Avalon-MM (memory mapped)  Little Endian  Control plane  Master interfa

© 2012 Altera Corporation—Confidential Memory Access for Deinterlacer 103 1 field write 4 field reads over 2 ports Motion Values 1080i/60 1080p/60 19

© 2012 Altera Corporation—Confidential Deinterlacing The problem - III  For best results, the interpolation must be performed in the correct directio

© 2012 Altera Corporation—Confidential Maths to the rescue!  Edge direction information may be gained in progressive video frames by using well known

© 2012 Altera Corporation—Confidential “calendar” sequence, bucket detail Dil II UDX 4 (ACDS 11.1) 106 Dil II UDX 5 (ACDS 12.1) Incorrect edge directi

© 2012 Altera Corporation—Confidential “flag” sequence Dil II UDX 4 (ACDS 11.1) 107 Dil II UDX 5 (ACDS 12.1) Incorrect edge direction at low angles ca

© 2012 Altera Corporation—Confidential moving zoneplate detail Dil II UDX 4 (ACDS 11.1) 108 Dil II UDX 5 (ACDS 12.1) Many incorrect edge direction dec

© 2012 Altera Corporation—Confidential What’s the cost ? Finally - ACDS 12.1 Release 109  Since the UDX5 release additional pipelining has been inclu

© 2012 Altera Corporation—Confidential f= (A+B)/2 g= (B+D)/2 h= (C+D)/2 e= (A+C)/2 VIP Suite IP: Scaling Algorithms Nearest Neighbor A A B B A A

© 2012 Altera Corporation—Confidential VIP Function Details Scaler Polyphase Algorithm  Data from multiple lines of the input image are assembled in

© 2012 Altera Corporation—Confidential Understanding Resource Tradeoff Scaler Polyphase Algorithm  Resource usage of a Nv x Nh taps scaling engine 

© 2012 Altera Corporation—Confidential Open, Low-overhead Interface Standard: Avalon Streaming (ST) Video 12 Open interface protocol for streaming vi

© 2012 Altera Corporation—Confidential Scaling: Not So Simple In Implementation  Video scaling is also subject to artifacts  Simple “stretching” is

© 2012 Altera Corporation—Confidential VIP Suite – Scaler II  Edge-adaptive scaling (v12.0)  Reduces blurriness while maintaining realism  Polypha

© 2012 Altera Corporation—Confidential Scaler II – Edge Adaptive  Available in ACDS 12.0  Standard polyphase Scaler II quality is ‘not bad’ so sign

© 2012 Altera Corporation—Confidential Scaler II – Edge Adaptive  2 additions to polyphase algorithm  Use two sets of coefficients, one for pixels

© 2012 Altera Corporation—Confidential Upscale (1)  For upscale  Some improvement in edge sharpness with reduced stepping 117 v11.1 v12.0

© 2012 Altera Corporation—Confidential Upscale (2)  For upscale  Some improvement in edge sharpness with reduced stepping 118 v11.1 v12.0

© 2012 Altera Corporation—Confidential Downscale (1)  For downscale  General improvement in the sharpness of the output image 119 v11.1 v12.0

© 2012 Altera Corporation—Confidential Downscale (2)  For downscale  General improvement in the sharpness of the output image 120 v11.1 v12.0

© 2012 Altera Corporation—Confidential Scaler II – Edge Adaptive Resource Usage Resource Usage on Arria V ALMs 3090 Memory Bits 313141 M10Ks 48 DSPs 1

© 2012 Altera Corporation—Confidential Exercise #6 Mixer, OSD, Ctrl Sync, and CRS IP

© 2012 Altera Corporation—Confidential Standard Interface Example 14 Avalon-MM Master AXI Slave Qsys Interconnect (simplified) AXI Master Avalon-MM Sl

© 2012 Altera Corporation—Confidential Lab 6: Mixer IP, OSD (and transparency), Ctrl Sync IP (for layer switching) and CRS IP 123

© 2012 Altera Corporation—Confidential 124 VIP Suite IP: Alpha Blending Mixer Select the number of layers to be mixed The level of blending is contro

© 2012 Altera Corporation—Confidential Control Synchronizer  When the control data packet of the next video frame changes  Control Synchronizer dete

© 2012 Altera Corporation—Confidential Control Synchronizer … in our case  The control synchronizer is used to ensure that register accesses to the s

© 2012 Altera Corporation—Confidential 127 VIP Suite IP: Chroma Resampler  This function allows you to change between 4:4:4, 4:2:2 and 4:2:0 samplin

© 2012 Altera Corporation—Confidential Exercise #7 VIP Reference Designs

© 2012 Altera Corporation—Confidential Reference Design: UDX6 on Arria-V starter kit Colour Space ConverterColour Space ConverterFrame ReaderInterlace

© 2012 Altera Corporation—Confidential UDX5 in S4GX 130

© 2012 Altera Corporation—Confidential UDX 5.1.3 (available now)  2 channels of 1080p60 processing featuring:  Motion Adaptive Deinterlacing  Edge

© 2012 Altera Corporation—Confidential 4K Upscale 3  Deinterlace and up-scale from SD to 3840x2160p60 (QFHD)  1x 3G SDI Input, 4 x 3G SDI Outputs 

© 2012 Altera Corporation—Confidential Qsys Design Flow 15  Create Quartus® II project  Create Qsys project from Quartus II  Add IP blocks to the

© 2012 Altera Corporation—Confidential 4K Upscale 3 Block Diagram Clipper ACLine BufferScaler ACClipper ACLine BufferScaler ACClipper ACLine BufferSca

© 2012 Altera Corporation—Confidential 4K Upscale 2.0 (available now)  1 channel of QFHD processing featuring:  Motion Adaptive Deinterlacing  Edge

© 2012 Altera Corporation—Confidential Introducing: A larger & powerful NEEK with 5 Mega Digital Camera VEEK Terasic’s Video & Embedded Eva

© 2012 Altera Corporation—Confidential How to Equip with VEEK  Download VIP Suite designs  Two type of files  Design files  SD card files  www.al

© 2012 Altera Corporation—Confidential Demo # 1 – Simple Format Conversion 137 Input Processing Output composite de-interlaced video /w txt overl

© 2012 Altera Corporation—Confidential Demo # 2 – Camera Video Scaling Input Processing Output Camera on VEEK Camera Input/w txt overlay (Camera

© 2012 Altera Corporation—Confidential Demo # 3 – Picture in Picture  2 video inputs  Composite, Camera  Key Video Functions  Deinterlacer, Scaler

© 2012 Altera Corporation—Confidential Demo # 4 – Multi-view  3 video inputs  Composite, Camera, DVI  1 DVI output with multiview  Key Video Funct

© 2012 Altera Corporation—Confidential Summary

© 2012 Altera Corporation—Confidential 142 Altera Technical Support over worldwide  Reference Quartus II software on-line help  Quartus II Handbook

© 2012 Altera Corporation—Confidential Use Qsys to Build VIP Systems  Start with a New Quartus II Project 16

© 2012 Altera Corporation—Confidential 143 Axios Technical Support in Korea  FAE support: 031-776-9888  Altera town café : http://cafe.naver.com/alt

© 2012 Altera Corporation—Confidential Learn More Through Technical Training 144 With Altera's instructor-led training courses, you can:  Le

© 2012 Altera Corporation—Confidential 145

© 2012 Altera Corporation—Confidential Thank You!

© 2012 Altera Corporation—Confidential Start a New System in Qsys 17

© 2012 Altera Corporation—Confidential Add the VIP Components 18

© 2012 Altera Corporation—Confidential Add the VIP Components 19

© 2012 Altera Corporation—Confidential Add Custom Components  Map component signals to Qsys Interface types and Signal Names on the Signals tab 20

© 2012 Altera Corporation—Confidential Connect the Components 21 Click the open dot to make connection

© 2012 Altera Corporation—Confidential Agenda  Altera Video Design Framework  Exercise #0 : VIP components & Introduction  Exercise #1 : Avalon

© 2012 Altera Corporation—Confidential Connect the Components 22

© 2012 Altera Corporation—Confidential 23

© 2012 Altera Corporation—Confidential Compile & Integrate Design 24 Qsys Quartus II Eclipse Development Kits

© 2012 Altera Corporation—Confidential Summary  Altera® video design framework enables rapid development  Mix & match existing IP - Leverage Al

© 2012 Altera Corporation—Confidential Exercise #0 VIP components & Introduction

© 2012 Altera Corporation—Confidential VIP Components Required for Lab 1  Test Pattern Generator  Clocked Video Output  Video Trace Monitor  Trace

© 2012 Altera Corporation—Confidential Lab 1: Video output and debug  Build basic VIP system in Qsys and debug 28

© 2012 Altera Corporation—Confidential VIP Function Details Generate a color bar or solid color test pattern 29

© 2012 Altera Corporation—Confidential VIP Function Details Clock Video Output  The Clocked Video Output MegaCore function converts from Avalon-ST Vi

© 2012 Altera Corporation—Confidential VIP Function Details Video Trace Monitor 31 System Console visualisation FPGA Video IP Video IP Av-ST V

© 2012 Altera Corporation—Confidential Altera Video Design Framework

© 2012 Altera Corporation—Confidential 32 The SystemConsole Framework  Extensible framework for moving data between the FPGA and the host  Trace fea

© 2012 Altera Corporation—Confidential QSys Debug System: Insert Monitors 33  Insert Avalon-ST Video Monitor(s) in the video datapath  Connect Monit

© 2012 Altera Corporation—Confidential Trace Table View 34 Live trace of selectable video control and data packets Av-ST Video protocol • Video contro

© 2012 Altera Corporation—Confidential Trace Table View – Pixel Data [Beta feature] 35 Av-ST Video protocol • Video data packet information Data f

© 2012 Altera Corporation—Confidential Exercise #1 Avalon Video Streaming Interface

© 2012 Altera Corporation—Confidential Interface Types  Avalon Streaming (ST) Video interface  Avalon Memory-Mapped (MM) interface 37 Clocked Video

© 2012 Altera Corporation—Confidential Avalon Streaming (ST) Video  Avalon-ST Video protocol is a packet-oriented way to send video and control data

© 2012 Altera Corporation—Confidential Video Data Packet  Sent per pixel in raster scan order  Static Parameters:  Bit per pixel per color p

© 2012 Altera Corporation—Confidential Color Pattern 40  Each color plane sample maps to Avalon-ST symbol  Symbol Transmission Order  Horizont

© 2012 Altera Corporation—Confidential Recommended Color Patterns 41 4:2:0 YCbCr 4:2:2 YCbCr 4:4:4 YCbCr RGB Sequence Parallel Recommended

© 2012 Altera Corporation—Confidential 4 Altera Video Framework  Building block IP cores to speed up development  Open, low-overhead interface stan

© 2012 Altera Corporation—Confidential Control Data Packet - Definitions 42 Interlacing Height Width Component 4 Number of lines in a frame o

© 2012 Altera Corporation—Confidential Control Data Packets - Examples Parameters Description Type Width Height Interlacing 15 1920 1080 0011 T

© 2012 Altera Corporation—Confidential Verification

© 2012 Altera Corporation—Confidential 45 Avalon-ST Video Verification IP suite – What is it?  A SystemVerilog class library  Released as part of Qu

© 2012 Altera Corporation—Confidential Avalon Streaming Verification Test Environment 46

© 2012 Altera Corporation—Confidential 47 Three types of classes  1 Video “items”  Pixels  Video data packets  Avalon-ST Video Control packets  A

© 2012 Altera Corporation—Confidential 48 Class type 1: Avalon-ST Video packet items  Easy to use :  function c_pixel pop_pixel();  function c_pixe

© 2012 Altera Corporation—Confidential 49 Class type 2 : Bus Functional Models - Source & sink  Randomized Avalon-ST Video at pin level 

© 2012 Altera Corporation—Confidential 50 File I/O test  Tests a user’s DUT with their own video files  Enables easy test of Avalon-ST video complia

© 2012 Altera Corporation—Confidential Exercise #2 Run-time Configuration

© 2012 Altera Corporation—Confidential 5 Video Image Processing (VIP) Suite BT656/1120, DVI  Avalon ST video Scaler / Scaler II Chroma resampler Alp

© 2012 Altera Corporation—Confidential Lab 2: Custom IP Verification and Insertion  Part 1:  Part 2: 52

© 2012 Altera Corporation—Confidential Run-Time Control 53  System Interconnect Fabric Scaler S i o Nios II Processor M CSC i o Nios II Process

© 2012 Altera Corporation—Confidential Control Registers Address Register Description 0 Go  Bit zero of this register is the Go bit, all other bi

© 2012 Altera Corporation—Confidential Updating Control Registers  Custom logic or a Nios II processor can be used to run-time configure the VIP core

© 2012 Altera Corporation—Confidential Software API  Altera provides a set of VIP APIs (C++) in the following reference designs:  Video Processing R

© 2012 Altera Corporation—Confidential Example – Run Time Configuration 57 CVO Nios II Processor CVI High Performance DDR2 Memory Controller Deinterla

© 2012 Altera Corporation—Confidential Example – Main program 58 CVO Nios II Processor CVI High Performance DDR2 Memory Controller Deinterlacer (MA) F

© 2012 Altera Corporation—Confidential Run-time Control of the Video Chain  Build video signal chains that can be updated on-the-fly without changing

© 2012 Altera Corporation—Confidential Run-time Control of the Video Chain  Most video functions permit run-time control of some aspects of their b

© 2012 Altera Corporation—Confidential Nios II Controls the Video PiP 62

© 2012 Altera Corporation—Confidential 7 Video Development Kits Cyclone III FPGA Stratix ® IV FPGA, Arria ® V FPGA Arria II GX FPGA Cyclone ® IV FP

© 2012 Altera Corporation—Confidential Exercise #3 Switch & clock/frame locking

© 2012 Altera Corporation—Confidential Lab 3: Switch IP, clock locking and frame locking 64

© 2012 Altera Corporation—Confidential Additional Components Required for Lab 3  Nios II Processor-Controlled Sub-System  Clocked Video Input VIP Co

© 2012 Altera Corporation—Confidential VIP Function Details Clock Video Input  The Clocked Video Input MegaCore function converts from clocked video

© 2012 Altera Corporation—Confidential Lab3: Video input (and debug) and NIOS run-time control 67

© 2012 Altera Corporation—Confidential Switch  Allows flexible routing of Avalon-ST Video  Can be used for muxing, de-muxing and crossbar switches 

© 2012 Altera Corporation—Confidential Bypassing An IP Core  Starting state  Video Input is connected to Out 1  In 1 is connected to Video Output 

© 2012 Altera Corporation—Confidential Avoiding Deadlock  Changing Switch 2, too early could result in deadlock  Solution is to poll until Switch 1

© 2012 Altera Corporation—Confidential Genlock  “Sync generator locking” is the process of aligning video outputs to the timing of a reference input

© 2012 Altera Corporation—Confidential Clock Locking – RX Clock 72 Sync SeparatorCVI CVOSDI TXsofsof_lockedvid_clkrefclk_divfhvrx_clkFPGAPLL74.25 MHz1

© 2012 Altera Corporation—Confidential Traditional System Design  Components in system use different interfaces to communicate (some standard, some n

© 2012 Altera Corporation—Confidential Clock Locking – VCXO 73 Divider +-Charge PumpPFDVCXOFeedback Divider27MHzPLLSync SeparatorCVI CVOSDI TXsofsof_l

© 2012 Altera Corporation—Confidential Frame Locking  Input and output frames must have the same frame rate  Frame rate is the number of frames per

© 2012 Altera Corporation—Confidential Frame Locking - Example  NTSC  Interlaced @ 29.97 fps  Deinterlace to progressive 480p @ 59.94 fps  13.5 M

© 2012 Altera Corporation—Confidential Clocked Video Output - Locking  Locking procedure  Compares sof signal to its internal sof  post-sof gap and

© 2012 Altera Corporation—Confidential Exercise #4 Frame buffer, FRC and Calculating Memory Bandwidth

© 2012 Altera Corporation—Confidential Lab 4: Ext Memory, Frame Buffer IP and Frame Rate Conversion 78

© 2012 Altera Corporation—Confidential Frame Buffer 79

© 2012 Altera Corporation—Confidential Frame Buffer 80 Double Buffering Frame Buffer Memory Writer Memory Reader System Interconnect Fabric External R

© 2012 Altera Corporation—Confidential Frame Rate Conversion  The Frame Buffer MegaCore performs frame rate conversion  It has two modes:  Triple

© 2012 Altera Corporation—Confidential Triple Buffer Mode  Output 1 shows frame repeating  Output 2 shows frame dropping  Output 3 shows locked fra

© 2012 Altera Corporation—Confidential Automatic Interconnect Generation  Avoids error-prone integration  Saves development time with automatic log

© 2012 Altera Corporation—Confidential External Memory Calculation  Calculate worst case external memory requirements of the system:  Compute worst

© 2012 Altera Corporation—Confidential External Memory Bandwidth Calculation 84 CVO Nios II Processor CVI High Performance DDR2 Memory Controller Dein

© 2012 Altera Corporation—Confidential Understanding External Memory Requirement Deinterlacer (motion adaptive)  Motion adaptive deinterlacer requir

© 2012 Altera Corporation—Confidential Understanding External Memory Requirement Deinterlacer (motion adaptive)  Motion adaptive deinterlacer requir

© 2012 Altera Corporation—Confidential External Memory Bandwidth Calculation Data Sample Packing Limiting factors:  20 bit YCbCr 4:2:2 pixels  10 b

© 2012 Altera Corporation—Confidential External Memory Bandwidth Calculation Worst-Case Bandwidth for Individual IPs Function Worst Case Input Format

© 2012 Altera Corporation—Confidential  Total System Bandwidth Requirement  The Arria V GX starter board when using the soft memory controller

© 2012 Altera Corporation—Confidential Efficient Memory Subsystem Architecture 90 CVO Nios II Processor CVI High Performance DDR2 Memory Controller De

© 2012 Altera Corporation—Confidential Multi-Port Front End Qsys Component (UDX Reference Design) 91 Design Requirements: • Adding Nios II and On Sc

© 2012 Altera Corporation—Confidential High Performance Controller II Features  Supports up to 1066 MHz DDR3 memory  Power management  Advanced ban

© 2012 Altera Corporation—Confidential Example Design in Qsys 10 M: Master S: Slave

© 2012 Altera Corporation—Confidential Exercise #5 Deinterlacing, Scaling

© 2012 Altera Corporation—Confidential Lab 5: Format Conversion (w/ Deinterlacer and Scaler IP) and CSC IP 94

© 2012 Altera Corporation—Confidential 95 First all odd lines scanned (1/60sec) then all even lines (1/60sec) presenting a full picture (1/30sec) All

© 2012 Altera Corporation—Confidential VIP Function Details Deinterlacer II (motion adaptive) NEW  Deinterlacer II  Supports 3:2 cadence detectio

© 2012 Altera Corporation—Confidential Deinterlacing The problem - I  Moving images in interlaced video streams cannot be simply woven together to pr

© 2012 Altera Corporation—Confidential Deinterlacing The problem - II  They can be better de-interlaced by interpolating the missing pixels using inf

© 2012 Altera Corporation—Confidential VIP Suite IP: Deinterlacing Algorithms Bob Interlacing Weave Interlacing: merge two neighboring field to a fra

© 2012 Altera Corporation—Confidential De-interlacing: Motion Adaptive Switches between Bob and Weave  Pixels are collected from the current field a

© 2012 Altera Corporation—Confidential Understanding External Memory Requirement Deinterlacer (motion adaptive)  ‘Motion bleed’ is optional  If sel

© 2012 Altera Corporation—Confidential Deinterlacer in the Frame Buffer Mode Deinterlacer can provide double or triple frame buffering in external RAM