Android上使用OpenGLES2.0显示YUV数据

论坛 › 期权论坛 › 脚本     
匿名网站用户   2020-12-19 17:01   11   0

转自：http://blog.csdn.net/ueryueryuery/article/details/17608185


在Android上用OpenGLES来显示YUV图像，之所以这样做，是因为：

1.Android本身也不能直接显示YUV图像，YUV转成RGB还是必要的；

2.YUV手动转RGB会占用大量的CPU资源，如果以这样的形式播放视频，手机会很热，所以我们尽量让GPU来做这件事；

3.OpenGLES是Android集成到自身框架里的第三方库，它有很多的可取之处。


博主的C/C++不是很好，所以整个过程是在Java层实现的，大家见笑，我主要参考（但不限于）以下文章，十分感谢这些朋友的分享：

1. http://blog.csdn.net/xiaoguaihai/article/details/8672631

2.http://chenshun87.blog.163.com/blog/static/18859389201232011727615/

3.http://blog.csdn.net/ypist/article/details/8950903

4.http://blog.csdn.net/wanglang3081/article/details/8480281

5.http://blog.csdn.net/xdljf/article/details/7178620


一、首先我先说一下这个解决方案是怎么运行的，给大家一个概念

1.显示在哪 -> GLSurfaceVIew

2.谁来把数据贴到GLSurfaceVIew上 -> Renderer

3.谁来负责YUV数据转换成RGB -> GL中的Program/Shader

一句话说明白就是：GL的Program/Shader把用户传过来的YUV数据，转换成RGB数据后，通过Renderer贴在GLSurfaceView上。


二、怎么检查你的手机是不是支持GLES2.0呢，使用下面的代码段就行了：

[java] view plaincopy在CODE上查看代码片派生到我的代码片
  1. public static boolean detectOpenGLES20(Context context) {
  2. ActivityManager am = (ActivityManager) context.getSystemService(Context.ACTIVITY_SERVICE);
  3. ConfigurationInfo info = am.getDeviceConfigurationInfo();
  4. return (info.reqGlEsVersion >= 0x20000);
  5. }
一般的手机，都是会支持GLES2.0的，大家不必担心。


三、开搞


A 先要有一个GLSurfaceView，把它放入你的布局中就好了。

找到这个家伙，对它进行简单的设置，并为它设置一个Renderer。

Renderer的作用就是在GLSurfaceView上画出图像。

[java] view plaincopy在CODE上查看代码片派生到我的代码片
  1. mGLSurface = (GLFrameSurface) findViewById(R.id.glsurface);
  2. mGLSurface.setEGLContextClientVersion(2);
  3. mGLFRenderer = new GLFrameRenderer(this, mGLSurface);
  4. mGLSurface.setRenderer(mGLFRenderer);

B 再就是看下GLFrameRenderer怎么来写了

[java] view plaincopy在CODE上查看代码片派生到我的代码片
  1. public class GLFrameRenderer implements Renderer {
  3. private ISimplePlayer mParentAct; //请无视之
  4. private GLSurfaceView mTargetSurface;
  5. private GLProgram prog = new GLProgram(0);
  6. private int mVideoWidth = -1, mVideoHeight = -1;
  7. private ByteBuffer y;
  8. private ByteBuffer u;
  9. private ByteBuffer v;
  11. public GLFrameRenderer(ISimplePlayer callback, GLSurfaceView surface) {
  12. mParentAct = callback; //请无视之
  13. mTargetSurface = surface;
  14. }
  16. @Override
  17. public void onSurfaceCreated(GL10 gl, EGLConfig config) {
  18. Utils.LOGD("GLFrameRenderer :: onSurfaceCreated");
  19. if (!prog.isProgramBuilt()) {
  20. prog.buildProgram();
  21. Utils.LOGD("GLFrameRenderer :: buildProgram done");
  22. }
  23. }
  25. @Override
  26. public void onSurfaceChanged(GL10 gl, int width, int height) {
  27. Utils.LOGD("GLFrameRenderer :: onSurfaceChanged");
  28. GLES20.glViewport(0, 0, width, height);
  29. }
  31. @Override
  32. public void onDrawFrame(GL10 gl) {
  33. synchronized (this) {
  34. if (y != null) {
  35. // reset position, have to be done
  36. y.position(0);
  37. u.position(0);
  38. v.position(0);
  39. prog.buildTextures(y, u, v, mVideoWidth, mVideoHeight);
  40. GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
  41. GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
  42. prog.drawFrame();
  43. }
  44. }
  45. }
  47. /**
  48. * this method will be called from native code, it happens when the video is about to play or
  49. * the video size changes.
  50. */
  51. public void update(int w, int h) {
  52. Utils.LOGD("INIT E");
  53. if (w > 0 && h > 0) {
  54. if (w != mVideoWidth && h != mVideoHeight) {
  55. this.mVideoWidth = w;
  56. this.mVideoHeight = h;
  57. int yarraySize = w * h;
  58. int uvarraySize = yarraySize / 4;
  59. synchronized (this) {
  60. y = ByteBuffer.allocate(yarraySize);
  61. u = ByteBuffer.allocate(uvarraySize);
  62. v = ByteBuffer.allocate(uvarraySize);
  63. }
  64. }
  65. }
  67. mParentAct.onPlayStart(); //请无视之
  68. Utils.LOGD("INIT X");
  69. }
  71. /**
  72. * this method will be called from native code, it's used for passing yuv data to me.
  73. */
  74. public void update(byte[] ydata, byte[] udata, byte[] vdata) {
  75. synchronized (this) {
  76. y.clear();
  77. u.clear();
  78. v.clear();
  79. y.put(ydata, 0, ydata.length);
  80. u.put(udata, 0, udata.length);
  81. v.put(vdata, 0, vdata.length);
  82. }
  84. // request to render
  85. mTargetSurface.requestRender();
  86. }
  87. }

代码很简单，Renderer主要处理这么几个事：

1.Surface create的时候，我初始化了一些需要用到的Program/Shader，因为马上就要用到它们了；

2.Surface change的时候，重置一下画面；

3.onDrawFrame()时，把数据真正地“画”上去；

4.至于两个update方法，是用来把图像的宽高/数据传过来的。


C 看GLProgram是怎么写的，它的作用是向Renderer提供计算单元，你所有对数据的处理，都在这儿了。

[java] view plaincopy在CODE上查看代码片派生到我的代码片
  1. public boolean isProgramBuilt() {
  2. return isProgBuilt;
  3. }
  5. public void buildProgram() {
  6. createBuffers(_vertices, coordVertices);
  7. if (_program <= 0) {
  8. _program = createProgram(VERTEX_SHADER, FRAGMENT_SHADER);
  9. }
  10. Utils.LOGD("_program = " + _program);
  12. /*
  13. * get handle for "vPosition" and "a_texCoord"
  14. */
  15. _positionHandle = GLES20.glGetAttribLocation(_program, "vPosition");
  16. Utils.LOGD("_positionHandle = " + _positionHandle);
  17. checkGlError("glGetAttribLocation vPosition");
  18. if (_positionHandle == -1) {
  19. throw new RuntimeException("Could not get attribute location for vPosition");
  20. }
  21. _coordHandle = GLES20.glGetAttribLocation(_program, "a_texCoord");
  22. Utils.LOGD("_coordHandle = " + _coordHandle);
  23. checkGlError("glGetAttribLocation a_texCoord");
  24. if (_coordHandle == -1) {
  25. throw new RuntimeException("Could not get attribute location for a_texCoord");
  26. }
  28. /*
  29. * get uniform location for y/u/v, we pass data through these uniforms
  30. */
  31. _yhandle = GLES20.glGetUniformLocation(_program, "tex_y");
  32. Utils.LOGD("_yhandle = " + _yhandle);
  33. checkGlError("glGetUniformLocation tex_y");
  34. if (_yhandle == -1) {
  35. throw new RuntimeException("Could not get uniform location for tex_y");
  36. }
  37. _uhandle = GLES20.glGetUniformLocation(_program, "tex_u");
  38. Utils.LOGD("_uhandle = " + _uhandle);
  39. checkGlError("glGetUniformLocation tex_u");
  40. if (_uhandle == -1) {
  41. throw new RuntimeException("Could not get uniform location for tex_u");
  42. }
  43. _vhandle = GLES20.glGetUniformLocation(_program, "tex_v");
  44. Utils.LOGD("_vhandle = " + _vhandle);
  45. checkGlError("glGetUniformLocation tex_v");
  46. if (_vhandle == -1) {
  47. throw new RuntimeException("Could not get uniform location for tex_v");
  48. }
  50. isProgBuilt = true;
  51. }
  53. /**
  54. * build a set of textures, one for Y, one for U, and one for V.
  55. */
  56. public void buildTextures(Buffer y, Buffer u, Buffer v, int width, int height) {
  57. boolean videoSizeChanged = (width != _video_width || height != _video_height);
  58. if (videoSizeChanged) {
  59. _video_width = width;
  60. _video_height = height;
  61. Utils.LOGD("buildTextures videoSizeChanged: w=" + _video_width + " h=" + _video_height);
  62. }
  64. // building texture for Y data
  65. if (_ytid < 0 || videoSizeChanged) {
  66. if (_ytid >= 0) {
  67. Utils.LOGD("glDeleteTextures Y");
  68. GLES20.glDeleteTextures(1, new int[] { _ytid }, 0);
  69. checkGlError("glDeleteTextures");
  70. }
  71. // GLES20.glPixelStorei(GLES20.GL_UNPACK_ALIGNMENT, 1);
  72. int[] textures = new int[1];
  73. GLES20.glGenTextures(1, textures, 0);
  74. checkGlError("glGenTextures");
  75. _ytid = textures[0];
  76. Utils.LOGD("glGenTextures Y = " + _ytid);
  77. }
  78. GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _ytid);
  79. checkGlError("glBindTexture");
  80. GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, _video_width, _video_height, 0,
  81. GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, y);
  82. checkGlError("glTexImage2D");
  83. GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
  84. GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
  85. GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
  86. GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
  88. // building texture for U data
  89. if (_utid < 0 || videoSizeChanged) {
  90. if (_utid >= 0) {
  91. Utils.LOGD("glDeleteTextures U");
  92. GLES20.glDeleteTextures(1, new int[] { _utid }, 0);
  93. checkGlError("glDeleteTextures");
  94. }
  95. int[] textures = new int[1];
  96. GLES20.glGenTextures(1, textures, 0);
  97. checkGlError("glGenTextures");
  98. _utid = textures[0];
  99. Utils.LOGD("glGenTextures U = " + _utid);
  100. }
  101. GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _utid);
  102. GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, _video_width / 2, _video_height / 2, 0,
  103. GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, u);
  104. GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
  105. GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
  106. GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
  107. GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
  109. // building texture for V data
  110. if (_vtid < 0 || videoSizeChanged) {
  111. if (_vtid >= 0) {
  112. Utils.LOGD("glDeleteTextures V");
  113. GLES20.glDeleteTextures(1, new int[] { _vtid }, 0);
  114. checkGlError("glDeleteTextures");
  115. }
  116. int[] textures = new int[1];
  117. GLES20.glGenTextures(1, textures, 0);
  118. checkGlError("glGenTextures");
  119. _vtid = textures[0];
  120. Utils.LOGD("glGenTextures V = " + _vtid);
  121. }
  122. GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _vtid);
  123. GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, _video_width / 2, _video_height / 2, 0,
  124. GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, v);
  125. GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
  126. GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
  127. GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
  128. GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
  129. }
  131. /**
  132. * render the frame
  133. * the YUV data will be converted to RGB by shader.
  134. */
  135. public void drawFrame() {
  136. GLES20.glUseProgram(_program);
  137. checkGlError("glUseProgram");
  139. GLES20.glVertexAttribPointer(_positionHandle, 2, GLES20.GL_FLOAT, false, 8, _vertice_buffer);
  140. checkGlError("glVertexAttribPointer mPositionHandle");
  141. GLES20.glEnableVertexAttribArray(_positionHandle);
  143. GLES20.glVertexAttribPointer(_coordHandle, 2, GLES20.GL_FLOAT, false, 8, _coord_buffer);
  144. checkGlError("glVertexAttribPointer maTextureHandle");
  145. GLES20.glEnableVertexAttribArray(_coordHandle);
  147. // bind textures
  148. GLES20.glActiveTexture(_textureI);
  149. GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _ytid);
  150. GLES20.glUniform1i(_yhandle, _tIindex);
  152. GLES20.glActiveTexture(_textureII);
  153. GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _utid);
  154. GLES20.glUniform1i(_uhandle, _tIIindex);
  156. GLES20.glActiveTexture(_textureIII);
  157. GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, _vtid);
  158. GLES20.glUniform1i(_vhandle, _tIIIindex);
  160. GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
  161. GLES20.glFinish();
  163. GLES20.glDisableVertexAttribArray(_positionHandle);
  164. GLES20.glDisableVertexAttribArray(_coordHandle);
  165. }
  167. /**
  168. * create program and load shaders, fragment shader is very important.
  169. */
  170. public int createProgram(String vertexSource, String fragmentSource) {
  171. // create shaders
  172. int vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexSource);
  173. int pixelShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentSource);
  174. // just check
  175. Utils.LOGD("vertexShader = " + vertexShader);
  176. Utils.LOGD("pixelShader = " + pixelShader);
  178. int program = GLES20.glCreateProgram();
  179. if (program != 0) {
  180. GLES20.glAttachShader(program, vertexShader);
  181. checkGlError("glAttachShader");
  182. GLES20.glAttachShader(program, pixelShader);
  183. checkGlError("glAttachShader");
  184. GLES20.glLinkProgram(program);
  185. int[] linkStatus = new int[1];
  186. GLES20.glGetProgramiv(program, GLES20.GL_LINK_STATUS, linkStatus, 0);
  187. if (linkStatus[0] != GLES20.GL_TRUE) {
  188. Utils.LOGE("Could not link program: ", null);
  189. Utils.LOGE(GLES20.glGetProgramInfoLog(program), null);
  190. GLES20.glDeleteProgram(program);
  191. program = 0;
  192. }
  193. }
  194. return program;
  195. }
  197. /**
  198. * create shader with given source.
  199. */
  200. private int loadShader(int shaderType, String source) {
  201. int shader = GLES20.glCreateShader(shaderType);
  202. if (shader != 0) {
  203. GLES20.glShaderSource(shader, source);
  204. GLES20.glCompileShader(shader);
  205. int[] compiled = new int[1];
  206. GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compiled, 0);
  207. if (compiled[0] == 0) {
  208. Utils.LOGE("Could not compile shader " + shaderType + ":", null);
  209. Utils.LOGE(GLES20.glGetShaderInfoLog(shader), null);
  210. GLES20.glDeleteShader(shader);
  211. shader = 0;
  212. }
  213. }
  214. return shader;
  215. }
  217. /**
  218. * these two buffers are used for holding vertices, screen vertices and texture vertices.
  219. */
  220. private void createBuffers(float[] vert, float[] coord) {
  221. _vertice_buffer = ByteBuffer.allocateDirect(vert.length * 4);
  222. _vertice_buffer.order(ByteOrder.nativeOrder());
  223. _vertice_buffer.asFloatBuffer().put(vert);
  224. _vertice_buffer.position(0);
  226. if (_coord_buffer == null) {
  227. _coord_buffer = ByteBuffer.allocateDirect(coord.length * 4);
  228. _coord_buffer.order(ByteOrder.nativeOrder());
  229. _coord_buffer.asFloatBuffer().put(coord);
  230. _coord_buffer.position(0);
  231. }
  232. }
  234. private void checkGlError(String op) {
  235. int error;
  236. while ((error = GLES20.glGetError()) != GLES20.GL_NO_ERROR) {
  237. Utils.LOGE("***** " + op + ": glError " + error, null);
  238. throw new RuntimeException(op + ": glError " + error);
  239. }
  240. }
  242. private static float[] squareVertices = { -1.0f, -1.0f, 1.0f, -1.0f, -1.0f, 1.0f, 1.0f, 1.0f, }; // fullscreen
  244. private static float[] coordVertices = { 0.0f, 1.0f, 1.0f, 1.0f, 0.0f, 0.0f, 1.0f, 0.0f, };// whole-texture
  246. private static final String VERTEX_SHADER = "attribute vec4 vPosition;\n" + "attribute vec2 a_texCoord;\n"
  247. + "varying vec2 tc;\n" + "void main() {\n" + "gl_Position = vPosition;\n" + "tc = a_texCoord;\n" + "}\n";
  249. private static final String FRAGMENT_SHADER = "precision mediump float;\n" + "uniform sampler2D tex_y;\n"
  250. + "uniform sampler2D tex_u;\n" + "uniform sampler2D tex_v;\n" + "varying vec2 tc;\n" + "void main() {\n"
  251. + "vec4 c = vec4((texture2D(tex_y, tc).r - 16./255.) * 1.164);\n"
  252. + "vec4 U = vec4(texture2D(tex_u, tc).r - 128./255.);\n"
  253. + "vec4 V = vec4(texture2D(tex_v, tc).r - 128./255.);\n" + "c += V * vec4(1.596, -0.813, 0, 0);\n"
  254. + "c += U * vec4(0, -0.392, 2.017, 0);\n" + "c.a = 1.0;\n" + "gl_FragColor = c;\n" + "}\n";

这里面代码比较复杂，我在这里稍作解释：

1.首先，buildProgram()目的要生成一个program，作用是用来将YUV->RGB，其中用到了2个shader（shader就相当于一个小运算器，它运行一段代码），第1个shader运行VERTEX_SHADER里的代码，目的是将坐标作为参数传入第2个shader；第2个shader来做YUV->RGB的运算。

2.buildTextures()是要生成3个贴图，分别为了显示R/G/B数据，三个贴图重合在一起，显示出来的就是彩色的图片。

3.drawFrame()是使用program来做运算，并真正去做画这个动作了。

回复
分享到 :
0 人收藏
返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

16级独孤
匿名网站用户

积分:1136255
帖子:227251
精华:0
+ 关注 私信
期权论坛 期权论坛
期权网络科技版权所有
关于我们
联系我们
加入我们
反馈问题
免责声明
积分充值
统一社会信用代码
积分规则
网站地图
爱文库
下属网站
官方
新浪微博
微信公众号
◆
下载
表情包
App下载
期权论坛 期权论坛

QQ咨询|关于我们|Archiver|手机版|小黑屋|( 辽ICP备15012455号-4 ) Powered by 期权论坛 X3.2 © 2001-2016 期权工具网&期权论坛 Inc.

发布
内容

下载期权论坛手机APP