This technology describes a video search engine for identifying scenes, events, locations and objects featured in on-line videos. The audio and visual components of a video stream are separately analyzed to create characteristic elements referred to as audio-visual atoms. Visual data is partitioned into short sections that are analyzed to identify objects and motion that yield information regarding the video’s contents. Multiple sections can be compared and combined to identify the most significant visual atoms. Characteristic sounds identified in the audio data are decomposed and simplified to create audio atoms. The combined audio-visual atoms can then form a characteristic signature of the video contents that can be compared to a database of video data to identify categories that best describe the video’s contents.
Analysis and automated sorting of videos using audio-visual atoms enables search engines to locate videos based upon their actual content rather than by user-provided descriptions, comments, or labels associated with the videos.
The technology was demonstrated by training the algorithm using part of Kodak’s consumer benchmark video collection and successfully using it to produce accurate content descriptions of the remaining videos in the collection.
Patent Issued (US 8,135,221)
Tech Ventures Reference: IR CU12269