[[teaching:ie0117:ie0117_proyectos_i_2014:proyecto_1:to_nclude_integrated_support_of_video_to_impressive| To include integrated support of video to Impressive]] ====Impressive==== ==== "To include integrated support of video to Impressive" : Project 1 ==== **José Rafael Arce Gamboa B20493** ==== Introduction == This project consists on modifying the source code of Impressive in such a way that it will allow the software to display a video in the moment that it comes integrated in the document of a presentation. In this way, Impressive will be capable to show a video included on a presentation on the same window that it is currently working whether it is included on the document, or if the document calls another application to show it. ==== General Objective ==== To include full support of integrated video to the presentations of PDF documents on Impressive ====Specific Objectives==== - Modify Impressive to allow the video display on the same window of the presentation, when the document contains a video file. - To adapt Impressive to show a video, even when the document calls another application to do so. ====Implementation==== Impressive is a program that displays presentations slides. It's source code is written in Python, and you can get more information about the software on it's web site [[http://impressive.sourceforge.net/index.php|Impressive]]. In order to download Impressive's source code, you must be sure you have the Subversion (SVN) repository. You can install it with the apt-get install command. $sudo apt-get install subversion The code will compile succesfully if you use the 156 version of SVN or bellow. You can change your SVN version by typing the following command. $svn update -r 156 And now, you can download the source code. $svn co http://svn.emphy.de/impressive/trunk/impressive According to Impressive's web site, in order to be able to modify Impressive's source code, you need to have the following external libraries: - OpenGL - PyGame - PyOpenGL - PIL - PDFTK - Xpdf - GhostScript - xdg-utils - PyWin32 - MPlayer All of them can be downloaded with the command. $aptitude install python-pygame python-opengl python-imaging pdftk poppler-utils xdg-utils mplayer Once the code is downloaded, you will have a directory named /impressive where there is a file named impressive_dev.py and a subdirectory /src. The file impressive_dev.py pulls the source files from /src and runs them. So, when you modify the source code you type the make command, and then execute the file impressive.py, which actually runs the programm. You need to add a file as an argument, so Impressive can show it. It should come with a testing file called demo.pdf which is made with latex beamer. Now, there arep two files on /src we are interested in modify, they are the file named control.py, which is the one that controlls what is done on a file, once it is shown, and the other is gldraw.py, which manages the operations of drawing the pages of the presentation, using the OpenGL library. But first, lets describe the code of a simple program that runs videos on screen, that was done to understand the working of a video player. The operation of this program is quite simple. It uses FFmpeg to decode a video file on many frames, and then uses PyGame to create a screen and show these frames one by one as a video. In order to do this, it was used pipeffmpeg, a fronted of FFmpeg that uses pipes to encode the video on frames in bitmap format, and send directly these bits of information to pygame to be able to show these frames. The details of instalation of pipeffmpeg can be found on it's web page [[https://github.com/kanryu/pipeffmpeg|pipeffmpeg]]. This program can also be downloaded using SVN and github. We are interested on using the class InputVideoStream which is the one that reads the video file and writes out the frames of it. Our program will be called reproductor_4.py, because it is the fourth version of it, and it's code is the following. #!/usr/bin/python import time import pygame from pygame.locals import * import sys import os import subprocess as sp import ctypes ##You have to import pipeffmpeg import pipeffmpeg from PIL import Image ##These are used to handle the BMP images from PIL import BmpImagePlugin import cStringIO as StringIO ###Here begins the function that plays video on a screen def reproducir(nombre): try: FRAMERATE = 30 pygame.init() clock = pygame.time.Clock() screen = pygame.display.set_mode((800, 600),pygame.FULLSCREEN) ##Here you use the class InputVideoStream() iv = pipeffmpeg.InputVideoStream() iv.open(nombre) for i, bmp in enumerate(iv.readframe()): ##You load each image image = pygame.image.load(StringIO.StringIO(bmp)) ##Then you show them on the screen screen.blit(image,(0,0)) del image pygame.display.flip() clock.tick(FRAMERATE) pygame.display.quit() except: pass return def main(): ##This is a main that tests a video file nombre = '/home/jose/impressive/IMG_2951.mpg' reproducir(nombre) if __name__=="__main__": main() The structure od the pygame code used to achive the working of this program was searched on the internet [[http://stackoverflow.com/questions/13643630/python-pygame-continuos-image-loading-fps|PyGame continous image loading]]. This code has a main function which uses the reproducir() function to show a file named '/home/jose/impressive/IMG_2951.mpg', although it can be used to show any other video file. Once it is done, this program should be saved in the same directory as the Impressive source code. We want to include these functions on the Impressive code, so it was first necessary to study how OpenGL library works, so it was written another simple program that uses the BMP files as OpenGL textures, and then shows them on screen. Here there is the source code. #!/usr/bin/env python from OpenGL.GL import * from OpenGL.GLU import * import pygame from pygame.locals import * import pipeffmpeg import reproductor_4 from PIL import Image from PIL import BmpImagePlugin import cStringIO as StringIO class Texture(): # simple texture class # designed for 32 bit png images (with alpha channel) def __init__(self,fileName): self.texID=0 self.LoadTexture(fileName) def LoadTexture(self,fileName): try: textureSurface = pygame.image.load(fileName) textureData = pygame.image.tostring(textureSurface, "RGBA", 1) self.texID=glGenTextures(1) glBindTexture(GL_TEXTURE_2D, self.texID) glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA, textureSurface.get_width(), textureSurface.get_height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, textureData ) glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR) glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR) except: print "can't open the texture: %s"%(fileName) def __del__(self): glDeleteTextures(self.texID) class Dibujar(): def resize(self,(width, height)): if height==0: height=1 glViewport(0, 0, width, height) glMatrixMode(GL_PROJECTION) glLoadIdentity() gluOrtho2D(-8.0, 8.0, -6.0, 6.0) glMatrixMode(GL_MODELVIEW) glLoadIdentity() def init(self, nombre): #set some basic OpenGL settings and control variables glShadeModel(GL_SMOOTH) glClearColor(0.0, 0.0, 0.0, 0.0) glClearDepth(1.0) glDisable(GL_DEPTH_TEST) glDisable(GL_LIGHTING) glDepthFunc(GL_LEQUAL) glHint(GL_PERSPECTIVE_CORRECTION_HINT, GL_NICEST) glEnable(GL_BLEND) self.tutorial_texture=Texture(nombre) self.demandedFps=30.0 self.done=False self.x,self.y=0.0 , 0.0 def draw(self): glBindTexture(GL_TEXTURE_2D,self.tutorial_texture.texID) glBegin(GL_QUADS) glTexCoord2f(0.0,1.0) glVertex2f(1.5, 5.0) glTexCoord2f(1.0,1.0) glVertex2f(6.75, 5.0) glTexCoord2f(1.0,0.0) glVertex2f(6.75, 0.0) glTexCoord2f(0.0,0.0) glVertex2f(1.5, 0.0) glEnd() ##This function is not important for this project, it allows to move a picture using the mouse buttons def Input(self): mpb=pygame.mouse.get_pressed() # mouse pressed buttons kpb=pygame.key.get_pressed() # keyboard pressed buttons msh=pygame.mouse.get_rel() # mouse shift if kpb[K_ESCAPE]: self.done=True if kpb[K_UP]: self.y+=0.1 if kpb[K_DOWN]: self.y-=0.1 if kpb[K_RIGHT]: self.x+=0.1 if kpb[K_LEFT]: self.x-=0.1 ##Testing main def main(): video_flags = OPENGL|DOUBLEBUF pygame.init() pygame.display.set_mode((640,480), video_flags) dibujo = Dibujar() dibujo.resize((640, 480)) iv = pipeffmpeg.InputVideoStream() iv.open('/home/jose/impressive/test.mp4') for i, bmp in enumerate(iv.readframe()): dibujo.init(StringIO.StringIO(bmp)) clock = pygame.time.Clock() dibujo.draw() pygame.display.flip() #limit fps clock.tick(dibujo.demandedFps) if __name__ == '__main__': main() This piece of code was done with the help of an example shown in this page. [[http://jason.gd|Ehttp://www.jason.gd/str/pokaz/pygame_pyopengl_2d]]. This program was called katana.py, because at the beginning it showed how to display an image file of a katana using textures. The Texture() class creates an OpenGL texture with an input image file, and Dibujar() class has a funciton draw() that takes this texture, and creates a rectangle on screen that contains the input file. We must notice that, the coordinates of the place that the video will be displayed on screen are determined on the draw() function, so we are facing the limitation that the current program can only play a video per page. The main() method is just to test this to display a video, using a code similar to the one found on reproductor_4.py So, we are going to save the katana.py program on the Impressive directory, because we are going to use it's methods later. Now we are ready to change Impressive's code. So, we go to the control.py file and do the following in the PlayVideo() function. # start video playback ############# PlayVideo() function def PlayVideo(video): if not video: return StopMPlayer() opts = ["-quiet", "-slave", \ "-monitorpixelaspect", "1:1", \ "-autosync", "100"] + \ MPlayerPlatformOptions if Fullscreen: opts += ["-fs"] else: try: opts += ["-wid", str(pygame.display.get_wm_info()['window'])] except KeyError: print >>sys.stderr, "Sorry, but Impressive only supports video on your operating system if fullscreen" print >>sys.stderr, "mode is used." VideoPlaying = False MPlayerProcess = None return if not isinstance(video, list): video = [video] try: # MPlayerProcess = subprocess.Popen([MPlayerPath] + opts + video, stdin=subprocess.PIPE) if MPlayerColorKey: glClear(GL_COLOR_BUFFER_BIT) pygame.display.flip() VideoPlaying = True except OSError: MPlayerProcess = None #################################################Ending of PlayVideo Function Actually, the only chage we have done is to comment the line that uses a subprocess statement to call MPlayer to display the video. If this line of code is present, then MPlayer uses a fullscreen to show the video, and then the presentation is disrupted. It is important to notice also, that we must use the info scripts to prepare Impressive to play the video. Info Scripts are text files with a Python dictionary that controlls the settings of the presentation of a particular file. For example, the info script of a PDF presentation with a video included should look like this. # -*- coding: iso-8859-1 -*- PageProps = { 1:{ 'title':"Pagina 1" }, 2:{ 'title': "Pagina 2", 'video': "/home/jose/impressive/IMG_2951.mpg" }, 3:{ 'title':"Pagina 3", 'video': "/home/jose/impressive/test.mp4" }, 4:{ 'title':"Pagina 4" }, } This corresponds to a four slide PDF presentation, which has a video on the second and third slide. This will ensure that Impressive can recognize the video and play it. Then we are going to change the gldraw.py file, which has the settings of the drawing of the pages on it. We are going to go to the DrawCurrentPage() function, which draws the complete image of the current page. This function is very large, but we are just goig to do two changes. At the very begining of the function, we are goig to ensure to have the following. # draw the complete image of the current page def DrawCurrentPage(dark=1.0, do_flip=True): from PIL import Image from PIL import BmpImagePlugin import cStringIO as StringIO import pipeffmpeg import katana from OpenGL.GLU import * video = GetPageProp(Pcurrent, 'video') if VideoPlaying: return boxes = GetPageProp(Pcurrent, 'boxes') glClear(GL_COLOR_BUFFER_BIT) With this, we are importing the necesary modules, and we are adding a GetPageProp to recognize a video present on the page. We have to remember that the current program can only play one video per page, so we must add only one video on the info script. Then, we are going to wrtie an if statement of what the program should do if it finds a video on the Info Script of the corresponding page. So, we are goig to go write the following between the if Marking statement, and the # unapply the zoom transform comment. #Play video on screen if video is not None: ##It calls an instance of katana module dibujo = katana.Dibujar() dibujo.resize((900,480)) #It creates an instance of pipeffmpeg module iv = pipeffmpeg.InputVideoStream() iv.open(video) ##This cycle plays the video for i, bmp in enumerate(iv.readframe()): EnableAlphaBlend() dibujo.init(StringIO.StringIO(bmp)) clock = pygame.time.Clock() dibujo.draw() DrawOverlays() pygame.display.flip() #limit fps clock.tick(dibujo.demandedFps) ###At the end of the video, the program jumps to the next page ###It was necesary to readjust the size of the pages again ###because resize function changes it video = None glViewport(-4767,-1915, 10900, 4600) glMatrixMode(GL_PROJECTION) glLoadIdentity() gluOrtho2D(-8.0, 8.0, -6.0, 6.0) glMatrixMode(GL_MODELVIEW) glLoadIdentity() TransitionTo(GetNextPage(Pcurrent, 1)) With this code, we are calling the functions of katana to draw the video frames on screen. At the end of the video, the program automatically jumps to the next page of the presentation, although if we want to watch the video again we can go back to the video using any of the Impressive's commands and the program will play the video one more time. At the end of this if statement, we must include some instructions to readjust the size of the following pages of the presentation, since it was necesary to use the resize function to readjust the size of the frames to fit in the video space of the screen. ==== Results==== The program runs succesfully, and for a given PDF presentation with videos inluded, Impressive opens a screen and displays the videos as soon as the page is turned on. Currently, the limitations of this program is that we still need to edit a Info Script in order to point out to Impressive that we want to show a video, and the program can only play one video on screen per page. If we want to display more than one video, we should edit the contents of the katana.py file. Here are a copy of the files created and edited.{{:ie0117_proyectos_i_2014:infoscript.png?200|}} {{:ie0117_proyectos_i_2014:gldrawpy1.png?200|}}{{:ie0117_proyectos_i_2014:classtexture.png?200|}} {{:ie0117_proyectos_i_2014:gldrawpy2.png?200|}}{{:ie0117_proyectos_i_2014:reproductor_4_codigo.png?200|}} ====Conclusions==== - It was possible to use PyGame, pipeffmpeg and OpenGL to write a program that display videos on screen. - The source code of Impressive was modified so it does not longer calls the MPlayer program to run the video on a fullscreen above the current slide. - The limitation is that you still need to edit your Info Scripts to use this feature, and the program can only play one video per page. ---- http://impressive.sourceforge.net/index.php\\ https://github.com/kanryu/pipeffmpeg\\ http://stackoverflow.com/questions/13643630/python-pygame-continuos-image-loading-fps\\ http://www.pythonforbeginners.com/os/subprocess-for-system-administrators\\ http://www.pygame.org/docs/ref/display.html\\ http://www.jason.gd/str/pokaz/pygame_pyopengl_2d\\