Vidyo Joins the Alliance for Open Media

aomlogo

Vidyo has been committed to the development of open standards since its inception in 2005. We have joined the ITU-T in 2006 as an Associate Member, and have participated in the development of both H.264 (AVC and SVC) and H.265 (HEVC) video coding standards. At the same time, we joined INCITS, the International Committee for Information Technology Standards, which is the central US forum for creating standards, and which designates the US delegation that represents the US in the Geneva-based International Standards Organization (ISO), including the well-known Motion Pictures Experts Group (MPEG). The ITU and MPEG have collaboratively developed both H.264 and H.265.  Our involvement in these efforts included the submission of technical proposals, donation of test video sequences, conformance bitstreams, as well as serving in editorial positions for various parts of these standards.

Vidyo has also been heavily involved in the IETF, which develops Internet-related standards including signaling, encryption, transport, etc. Vidyo’s personnel have co-authored numerous RFCs, and serve as co-chairs in relevant working groups. Vidyo has also been a member since 2010 of the International Multimedia Telecommunications Consortium (IMTC), which develops interoperability specifications and runs its annual SuperOp! interoperability testing events (now in its 21st year).

Likewise, we have been involved in the WebRTC standardization effort, both at the IETF as well as the video codec level. Vidyo has improved VP9 by collaborating with Google to add scalability, an important feature for high-quality videoconferencing.

Our involvement in these efforts stems from our belief that standards and interoperability are crucial for improving the quality that users obtain collectively from our industry. By raising the quality, we help the industry grow; by fostering standardization and working on interoperability we make it easier to incorporate innovations across the board and deliver improved solutions to customers and users.

The Alliance for Open Media is a brand new organization in the area of standardization, founded by some of today’s leading companies in technology and content distribution (including Google), with a goal of developing next-generation royalty-free media codecs for a rapidly transforming industry. Vidyo’s innovative server architecture for large-scale, very high quality video communication, on virtually any network and device, is inherently codec-agnostic: the choice of codec, from an infrastructure point of view, is purely a business decision. We are very excited to join the Alliance for Open Media and contribute to the development of the AV1 codec, particularly for real-time videoconferencing uses.

Google and Vidyo Bring VP9 to WebRTC

WebRTC600

Google just announced that the upcoming release of its Chrome browser, version 48, will have VP9 support for WebRTC.  This is an important milestone for WebRTC and Chrome for a number of reasons.  To appreciate why, it is useful to look a little bit at the history of video codec support in WebRTC as well as Google+ Hangouts, Google’s popular videoconferencing app.

HangoutsWhen Google+ Hangouts was originally launched, it used H.264 SVC, the scalable video coding extension of H.264 that Vidyo co-developed.  The use of scalability in the video codec is essential for implementing a radically different server design for multipoint video, able to deliver an unprecedented range of features: error resilience and localization, rate adaptation, personalized layout, cascading, etc.  More significantly, all this is done with extremely low delay (an order of magnitude less than a typical transcoding MCU), at a massive scale, and without any signal processing at the server.  Vidyo introduced this server design in 2008 in its patented VidyoRouter™ product.

Hangouts moved to use the open-source VP8 video codec in August 2013 so that, among other things, the client could eventually run over WebRTC as explained in interviews by senior Google management. As VP8 does not offer spatial scalability, a simpler version of it called simulcasting was used.  With this technique an endpoint produces multiple video streams at different bit rates and resolutions.  The server receives both, and selects which one to forward to receiving participants. (We later on coined the term “Selective Forwarding Unit”, or SFU, to describe the operation of these servers in a generic way.)

At the same time the transition to VP8 was being made, Vidyo and Google announced that Vidyo would be collaborating with Google to develop a scalable extension for the VP9 video codec as part of the WebRTC client open source project.  The goal is to combine the power of WebRTC with the benefits of scalability to be able to deliver the best possible experience for the end users.

VP9

The inclusion of VP9 in WebRTC in Chrome 48 marks an important milestone in this collaboration.  It is the first widely available software release of this new codec (outside a flag, across all Chrome platforms). The release incorporates the proposed standard RTP packetization for VP9, which was co-designed and developed with Vidyo and Google, and which supports the new scalability features.  Spatial and temporal scalability features in the codec itself are already in the WebM and WebRTC code repositories.

ChromeAs additional experience in deployed applications is obtained, and appropriate API support is made available at the WebRTC layer, we expect they will find their way into future versions of the Chrome browser in a backwards-compatible way.

Vidyo has utilized scalable coding to built a new server architecture for large-scale, very high quality video communication, on virtually any network and device.  This architecture excels in environments that require large scale and/or are hosted in the cloud.  It is also inherently codec-agnostic: the choice of the codec, from an infrastructure point of view, is purely a business decision.  As long as scalability is available, then the infrastructure can perform its magic.

We are very excited that we will soon be able to bring the proven benefits of this design to the WebRTC world. While this release is the first step, as the saying goes, the first step is half the journey.

Vidyo Surpasses 100 Patents and 10 years of Video Conferencing Innovation

Vidyo recently celebrated its 10-year anniversary. It’s amazing to contrast the range of products that we offer today, and the number of people that we reach every day through our software solutions with the first meetings of the early days. What started off as mere concepts on whiteboards and presentation slides has translated to technology that has changed our industry in very significant ways.

Vidyo-Infrastructure-Patents

Vidyo’s software-based approach limits the need for transcoding and associated infrastructure.

I could describe these changes in various terms – changes in product architectures, standards, industry best practices, and so on – but there is a great measurement of innovation that has the benefit of originating from objective third parties: patent offices. As it turns out, Vidyo recently celebrated the issuance of its 100th worldwide patent.

Vidyo was founded in order to build a disruptive, new architecture for large-scale, very high quality video communication, on virtually any network and device. From day one, as a new company in a space of considerable technological complexity, it sought to protect its innovations through patent applications filed in the US and elsewhere around the globe. These patent applications started producing results within a few years, with our first patent being issued in the US on September 22, 2009 (US Pat. Nr. 7,593,032) and the 100th being issued in the US on July 7, 2015 (US Pat. Nr. 9,078,004). As of today Vidyo has a total of 108 patents worldwide.

Vidyo-First-Patent

Vidyo’s First Patent issued in the United States – September 22, 2009

Vidyo-100-Patent

Vidyo’s 100th Patent issued in the United States – July 7, 2015

Vidyo’s patents span a wide range of subjects. Some relate directly to our products, some have been found essential to practice certain industry standards in video compression and communication, and some describe more strategic technology explorations. Several apply to more than one category.

Although each patent has its own history and background, I chose to highlight here two particular patent “families.” (By patent family we typically refer to an invention for which patents are being pursued at multiple jurisdictions around the world, and which may result even in multiple actual patents in the same jurisdiction.)

The first family happens to include our very first issued patent, US Pat. Nr. 7,593,032.  The title, “System and Method for a Conference Server Architecture for Low Delay and Distributed Conferencing Applications,” describes certain aspects of the operation of our VidyoRouter. The VidyoRouter is a cornerstone of Vidyo’s technical architecture, in that it takes advantage of scalable video coding to deliver an unprecedented range of features (error resilience and localization, rate adaptation, personalized layout, cascading, etc.) with extremely low delay (an order of magnitude less than a typical MCU) and at a massive scale – and all that without any signal processing at the server. The VidyoRouter’s design did not exist as a concept in the Internet Engineering Task Force (IETF) specifications when it was first introduced in our products in 2008.  For example, the RTP Topologies RFC, RFC 5117, published at the same time, did not foresee it. We have subsequently introduced the concept of the “Selective Forwarding Unit” (SFU), now incorporated into the RTP Topologies update (which should replace RFC 5117 soon), to cover this design using a generic, non-commercial term.

The second patent family that I would like to highlight includes US Pat. Nr. 9,077,964, “Systems and Methods for Error Resilience and Random Access in Video Communication Systems.” As the title suggests this patent deals with error resilience. It covers a key concept for achieving error resilience through the use of temporal scalability. The concept is very simple: by using temporal scalability, one merely has to protect the lowest temporal layer, typically one fourth or one half of the original pictures, rather than the entire set of pictures. Since these pictures are now at two or four times the temporal distance, the system can use retransmission to recover any parts of them that may be lost.  Using a design that allows the system to detect such loss with the very first packet that it correctly receives, one can achieve error resilience that is an order of magnitude higher than techniques that use non-scalable video. At Vidyo we decided that the parameters used in this technique would be of significant value to the entire industry and so proposed their inclusion in all RTP payload formats of relevant codecs: H.264 SVC (RFC 6190), H.265VP8, and VP9. We submitted intellectual property rights (IPR) disclosures at the IETF that ensure that people who implement these standards can use them*. At the same time, certain claims of the 9,077,964 patent and other Vidyo patents have also been found essential for implementing the H.264 SVC and H.265 codecs, and we have licensed the relevant claims through the corresponding patent pools managed by MPEG-LA, thus making it easier for people to make use of these standards as well.

These two patent families highlight two important dimensions of Vidyo’s technology development efforts, which form a continuous balancing act. Some innovations we maintain exclusively to Vidyo, as they are part of what makes Vidyo unique. Some others we believe are better suited to be incorporated into standards so that anybody can have access to them, thus benefitting the entire industry – both producers and users (ourselves included, of course!). As the industry matures further, one can expect more and more technical features to become standardized, as new ones come in to define differentiation. This has been the traditional development cycle of standardization and innovation for a long time, and one in which we expect to continue to be an active participant.

It’s been a fascinating journey – and a real privilege to be part of the team that is making it happen. On to the next 100!

*Please refer to the specific IPR disclosures for the precise legal framework surrounding these disclosures.

Deploying WebRTC: Straight Ahead, with Two Sharp Turns

It seems everybody will be showing up at the WebRTC III Conference and Expo this week. The event has quickly become the premier venue for all things related to WebRTC, and attracts hundreds of people to standing-room-only presentations. It is rare that a technology generates so much interest so early in its development phase.

A lot of the industry excitement is perfectly justified. Until now, video and audio communication was only possible using dedicated applications and devices. WebRTC will provide a standardized API to the functions of a real-time communication device, built into a web browser. This will decouple application development from video engine development and unleash web-based application development creativity into the real-time communications arena. Video and audio communication will now be integrated into applications in ways that were previously not possible. Although companies like Vidyo have already been creating APIs for their video engines, WebRTC is the first instance of an industry-standard API. The fact that it originates from W3C and IETF further solidifies its credentials. The elimination of the “download” step is also very important in several application deployment scenarios.

In the midst of the excitement, however, there are two things that need to be considered. First, putting an API around a video and audio engine does not mean that one has solved the problems of packet-based video and audio transport over the Internet. This includes congestion control, rate control, error resilience, error concealment, among many others. An API is, and should be, transparent with respect to all these issues, since it does not affect application design and implementation. The quality of experience that a user will get, however, depends on what’s under the hood.

Vidyo has demonstrated that scalable video coding is the way forward for high quality, real-time video over the Internet, and nearly all companies in the field have followed our lead. Our recent announcement with Google concerning the development of a scalable video extension for VP9 is further testament to the need for scalability on the video codec, regardless of its origin. An inherent benefit of the WebRTC architecture is that one will be able to use a better codec without changing any part of the application’s code.

The second consideration that WebRTC enthusiasts should take into account is that in order for WebRTC to fulfill its promise of ubiquitous use of video and audio communication in web applications, it must be possible to design servers that can support the scale of these video and audio applications. This is essential for multi-point applications. A WebRTC server application must be able to support hundreds of users to be economically sensible. Traditional transcoding servers (MCUs) that have been used in videoconferencing are very complex, expensive, introduce unacceptable delay, and have very poor scalability (number of simultaneous users). This is a well-known problem in the video conferencing industry, and one that Vidyo has successfully solved with its patented VidyoRouter architecture. The VidyoRouter performs no transcoding, and can support hundreds of users from a single 1 RU box. Introduced in 2008, together with the first-ever Scalable Video Coding (SVC) endpoint design, it brought video conferencing in line with any other network application. Today the VidyoWorks platform represents the best solution for supporting large-scale WebRTC deployments.

There is no doubt in my mind that WebRTC represents a great leap forward for video and audio communication. In fact, I am sure we will all be surprised by the range and reach of the applications that will be developed. All the elements are getting in place and the road ahead is clear for its deployment and widespread use. Application developers should realize, however, that WebRTC is an endpoint API. Architects must take into consideration all the additional elements that are required to create successful applications that can scale to large numbers of users, and provide the high quality that users expect. A scalable codec and a non-transcoding server are the two most important elements.

The excitement today about WebRTC is centered on the application development community. If we are to transfer it to the user community we must make sure that we deliver the best possible quality and experience to them. In the past, legacy videoconferencing systems failed to meet consumer expectations so users shied away from using it. Today we finally have the technical know-how to deliver superb video and audio quality everywhere. We must make sure that we use it; users will reject any application, or technology, that falls short regardless of how promising the architecture may be.

Alex drives the technical vision and direction for Vidyo and also represents the company on standardization committees and technical advisory boards. He is an award-winning researcher, bringing over 23 years of research experience in video compression and communications to his role at Vidyo. Prior to Vidyo he was an Associate Professor of Electrical Engineering at Columbia University. Alex has more than 100 publications, holds 64 patents (several of which are used in Blu-ray Disc, H.264/AVC, and ATSC digital television systems), and has served as the Editor of the MPEG-4 Systems specification, Co-Editor of the H.264 SVC Conformance specification, and Co-Editor of IETF’s RTP Payload Format for SVC.  He is a Member of the Boards of IMTC and of the UCI Forum, and co-chairs the IMTC SVC Activity Group as well as the UCI Forum’s SVC Technical Working Group.

 

A Brighter Future in Video Conferencing: UCI Forum Approves H.264/MPEG-4 Part 10 AVC and Scalable Video Coding (SVC) Modes Specification

Video conferencing technology has vastly improved over the last several years creating new market opportunities and opening a significantly larger addressable market compared with 2-3 years ago. As mentioned in Frost & Sullivan’s 2013 “Best Practices Research Report,” Vidyo has developed a platform that enabled this growth, and has benefited from the innovations it has brought to the market. Among them is H.264 Scalable Video Coding (SVC).

The recent approval of H.264/MPEG-4 Part 10 AVC and SVC Modes Version 1.0 specification from the Unified Communications Interoperability Forum (UCI Forum) is an important milestone for the entire industry.  It is the first step towards offering better video communication to customers by enabling them to deploy mixed-vendor solutions with confidence. The approved specification is also great news for Vidyo, as our customers have repeatedly asked for interoperability. The approval of this specification widens the doorway to seamless interoperability at the media level, as SVC is adopted by more and more vendors.

Vidyo has always been an advocate for SVC and, in fact, helped lay the initial groundwork for the original H.264 SVC specification. Vidyo engineers worked closely within standard development groups such as the ITU-T, MPEG, and the IETF, and even co-wrote parts of the SVC specifications. H.264 SVC technology folds seamlessly into today’s videoconferencing architectures and addresses several issues surrounding visual communications on unreliable networks.

H.264 SVC is an international standard. The large number of options and configuration choices, however, hinders interoperation between products from different vendors. The goal of the UCI Forum H.264 AVC & SVC Modes specification is to provide detailed guidance as to what operating modes and capabilities are required of both decoders and encoders, in order to enable interoperability. The specification that was just completed is the first step: it concerns the video bitstream format only and was developed as a joint effort of the membership of the H.264/SVC Task Group of UCI Forum. As a result, the H.264/SVC specification is not the product of a single vendor, but rather represents the consensus of several different H.264 SVC implementors.

As more companies realize H.264 SVC’s value in addressing market needs, you are sure to see the SVC adoption rise. In fact many companies have begun to adopt SVC in one form or another just this past year. Vidyo has been steadfast in its commitment to facilitating interoperability of the H.264 SVC standard for over six years.  Our VidyoRouter architecture, which features H.264 SVC and Adaptive Video Layering, is being embraced by more and more customers from a variety of markets due to the many unique benefits it offers: exceptional quality, highly scalable, affordable video conferencing, accessible via any network using any device, just for starters. We now have thousands of customers using Vidyo’s technology and our 3rd generation video communications products, who enjoy the benefits of this groundbreaking architecture every single day.

We are dedicated to continuing this pursuit into the future and to making this vital technology available to all. So keep an eye on H.264 SVC as more work is underway to enable interoperability at the transport and signaling levels.

For more information about the new H.264/SVC specification, click here.
For more information about SVC and Video Communications, click here.
For more information about UCI Forum, click here.

———————————————————————————————————–

Alex drives the technical vision and direction for Vidyo and also represents the company on standardization committees and technical advisory boards. He is an award-winning researcher, bringing over 23 years of research experience in video compression and communications to his role at Vidyo. Prior to Vidyo he was an Associate Professor of Electrical Engineering at Columbia University. Alex has more than 100 publications, holds 64 patents (several of which are used in Blu-ray Disc, H.264/AVC, and ATSC digital television systems), and has served as the Editor of the MPEG-4 Systems specification, Co-Editor of the H.264 SVC Conformance specification, and Co-Editor of IETF’s RTP Payload Format for SVC.  He is a Member of the Boards of IMTC and of the UCI Forum, and co-chairs the IMTC SVC Activity Group as well as the UCI Forum’s SVC Technical Working Group.

 

 

Raising our Telepresence Standards

Telepresence has been a word that has been very much in the news recently.

First, together with HD, it has been a big part of the push for next-generation videoconferencing systems throughout the industry. Second, it has – for a long time –  been the centerpiece of Cisco’s high-end video communication offerings leading up to the acquisition of Tandberg. Cisco’s TIP protocol (Telepresence Interoperability Protocol) has been transferred to the IMTC in order to comply with requirements set by regulators (Here’s the EU decision).

If you thought that these are all good indications that telepresence is here to stay, there is more.  The IMTC has formed a Telepresence Activity Group that studies telepresence systems and generates requirements. These requirements have also been submitted to the IETF for consideration.  IETF is the body that makes Internet network-related standards, including RTP, and SIP. These requirements will either result in work to be performed in existing IETF working groups, or they may even result in a completely new working group.

The IMTC has formed a Telepresence Activity Group that studies telepresence systems and generates requirements.

Finally, and this is really big news, the ITU (International Telecommunications Union) has just created a new working group on Telepresence Systems (officially, Question 5 of Study Group 16). The group was formed in July as a result of a proposal by the United States. ITU is the same body that develops video coding standards such as H.264 and signalling standards such as H.323.  The new group will, among other things, standardize the means for full interworking between telepresence systems.

All this activity is a sure indication that we will be seeing much more telepresence-related technology in the very near future.  Vidyo is very much involved in all this, and is very excited to see the high-end of the videoconferencing space grow so fast.  By offering the only architecture that enables seamless co-operation of users from practically any device, we are very excited to have interoperability standards that allow all systems to talk to each other.

Make sure you see our VidyoMobile ™ demonstration!

And don’t forget: in telepresence, it should not be business, it should be personal. 🙂

———————————————————————————————————–

Dr. Alex Eleftheriadis, is Chief Scientist and co-founder of Vidyo. Alex drives the technical vision and direction for Vidyo and also represents the company on standardization committees and technical advisory boards. He is an award-winning researcher, bringing over 19 years of research experience in video compression and communications to his role at Vidyo. Prior to Vidyo he was an Associate Professor of Electrical Engineering at Columbia University. Alex has more than 100 publications, holds 15 patents, has served as the Editor of the MPEG-4 Systems specification and Co-Editor of the H.264 SVC Conformance specification, and is currently Co-Editor of IETF’s RTP Payload Format for SVC.