1 00:00:01,489 --> 00:00:04,659 - In this final segment on OpenGL, 2 00:00:04,659 --> 00:00:08,301 we will be talking about texture mapping. 3 00:00:08,301 --> 00:00:12,731 We will add a texture of wood to the wooden floor, 4 00:00:12,731 --> 00:00:16,898 in order to make the mytest demo more interesting. 5 00:00:18,150 --> 00:00:23,032 This segment will correspond to the mytest3 program. 6 00:00:23,032 --> 00:00:25,632 Let me first show you a demo. 7 00:00:25,632 --> 00:00:28,643 Here I see that the only change is, 8 00:00:28,643 --> 00:00:31,240 that the floor instead of the being white, 9 00:00:31,240 --> 00:00:33,768 now has a wood texture with it. 10 00:00:33,768 --> 00:00:37,935 And I can toggle the texture on and off using the 't' key. 11 00:00:39,028 --> 00:00:43,639 I've defined a bunch of globals to set up the texture. 12 00:00:43,639 --> 00:00:47,389 woodtexture is 256 x 256 RGB texture map. 13 00:00:49,560 --> 00:00:53,666 I have a texture buffer, which is given by texNames, 14 00:00:53,666 --> 00:00:56,748 istex is a parameter for texturing, 15 00:00:56,748 --> 00:00:58,788 islight similarly for lighting, 16 00:00:58,788 --> 00:01:01,150 and I have these texturing and lighting flags 17 00:01:01,150 --> 00:01:04,326 to turn on and off, texturing and lighting. 18 00:01:04,326 --> 00:01:06,139 In the display routine, 19 00:01:06,139 --> 00:01:10,087 I initially set islight to zero to turn off lighting, 20 00:01:10,087 --> 00:01:13,402 except we'll turn it on later for the teapot. 21 00:01:13,402 --> 00:01:17,569 I set the istex flag to texturing, and I draw texture. 22 00:01:19,029 --> 00:01:21,119 So instead of just drawing an object, 23 00:01:21,119 --> 00:01:22,748 or drawing with color for the cubes, 24 00:01:22,748 --> 00:01:25,298 I draw with texture the floor, 25 00:01:25,298 --> 00:01:28,381 and I provided the texture name 0, 26 00:01:29,866 --> 00:01:32,199 which is the texture buffer. 27 00:01:33,731 --> 00:01:36,294 Finally, I set istex to zero, 28 00:01:36,294 --> 00:01:39,961 to ensure that other items are not textured. 29 00:01:43,052 --> 00:01:45,794 The keyboard has some simple toggles, 30 00:01:45,794 --> 00:01:49,683 most notably 't' for turning on and off texturing. 31 00:01:49,683 --> 00:01:52,602 After this, I have to queue up the display routine 32 00:01:52,602 --> 00:01:54,684 at glutPosRedisplay. 33 00:01:54,684 --> 00:01:56,872 's' turns on and off shading, 34 00:01:56,872 --> 00:01:58,932 which we'll not be talking about here, 35 00:01:58,932 --> 00:02:02,219 but you can experiment with it in the program. 36 00:02:02,219 --> 00:02:04,663 Let me show you the demo again, 37 00:02:04,663 --> 00:02:07,443 and I can just turn 't' on and off. 38 00:02:07,443 --> 00:02:09,142 Notice that I can do everything else. 39 00:02:09,142 --> 00:02:11,873 I can zoom in, zoom out. 40 00:02:11,873 --> 00:02:13,811 Let's back up a level, 41 00:02:13,811 --> 00:02:17,342 and talk about the idea of texture mapping. 42 00:02:17,342 --> 00:02:20,421 This really is one of the great inventions 43 00:02:20,421 --> 00:02:22,909 in computer graphics. 44 00:02:22,909 --> 00:02:24,932 For motivation, consider the geometry 45 00:02:24,932 --> 00:02:27,270 of this dinosaur shown here, 46 00:02:27,270 --> 00:02:31,025 and you want to be able to create an image of it, 47 00:02:31,025 --> 00:02:33,525 with realistic colors on it; 48 00:02:34,466 --> 00:02:37,946 which will be done by taking this texture on the right, 49 00:02:37,946 --> 00:02:40,645 and wrapping it on the dinosaur. 50 00:02:40,645 --> 00:02:44,312 Now, if I had to do this in a geometric way, 51 00:02:45,623 --> 00:02:48,932 I would have to make very small polygons, 52 00:02:48,932 --> 00:02:51,814 each of which is colored with the correct 53 00:02:51,814 --> 00:02:53,481 texture map element, 54 00:02:54,578 --> 00:02:58,434 and I would have to have a huge amount of geometry 55 00:02:58,434 --> 00:03:00,385 to represent the dinosaur. 56 00:03:00,385 --> 00:03:04,964 This would slow down my 3D Graphics rendering. 57 00:03:04,964 --> 00:03:07,372 However, with texture mapping, 58 00:03:07,372 --> 00:03:09,725 I can represent the 3D geometry 59 00:03:09,725 --> 00:03:11,824 at a relatively coarse scale, 60 00:03:11,824 --> 00:03:16,076 but I am now adding detail in an image-based way, 61 00:03:16,076 --> 00:03:20,243 having an image which encodes the fine-scale texture detail. 62 00:03:21,573 --> 00:03:25,773 And in this way I can still get by with coarse geometry. 63 00:03:25,773 --> 00:03:28,263 Texture mapping is a very important topic 64 00:03:28,263 --> 00:03:32,372 in practical computer graphics. In almost any application, 65 00:03:32,372 --> 00:03:36,292 almost all of the objects will end up being texture-mapped. 66 00:03:36,292 --> 00:03:39,545 In the real world, essentially all objects have 67 00:03:39,545 --> 00:03:41,063 some kind of texture to them, 68 00:03:41,063 --> 00:03:44,453 whether it's wood grain, whether it's the face texture, 69 00:03:44,453 --> 00:03:46,444 whether it is bricks. 70 00:03:46,444 --> 00:03:49,315 And this adds a level of image-based visual 71 00:03:49,315 --> 00:03:51,663 detail to scenes, which can make scenes 72 00:03:51,663 --> 00:03:54,386 look much more complicated and interesting, 73 00:03:54,386 --> 00:03:58,423 even if the geometry of the scene is very simple. 74 00:03:58,423 --> 00:04:01,474 It can be added in a fragment shader, 75 00:04:01,474 --> 00:04:04,025 and you can see in the left and right images here, 76 00:04:04,025 --> 00:04:06,205 the greater realism of the scene, 77 00:04:06,205 --> 00:04:10,395 once you've added the appropriate texture maps. 78 00:04:10,395 --> 00:04:13,433 Let's talk about setting up the texture. 79 00:04:13,433 --> 00:04:16,864 So we call this inittexture() function. 80 00:04:16,864 --> 00:04:18,894 This function just reads in 81 00:04:18,894 --> 00:04:22,093 a portable pixel map, or ppm file. 82 00:04:22,093 --> 00:04:24,763 PPM files are not very common nowadays, 83 00:04:24,763 --> 00:04:27,711 so you can replace this with an image read 84 00:04:27,711 --> 00:04:30,470 for your favorite file format if you want. 85 00:04:30,470 --> 00:04:32,443 But let me just explain it. 86 00:04:32,443 --> 00:04:34,875 You take in a pointer to the file name, 87 00:04:34,875 --> 00:04:38,933 and the program corresponding to the shader program. 88 00:04:38,933 --> 00:04:42,102 This is just old style C code for reading a file, 89 00:04:42,102 --> 00:04:44,812 fp is the file pointer, 90 00:04:44,812 --> 00:04:48,645 and the ppm format starts with a line 91 00:04:50,277 --> 00:04:53,413 which usually says P6, then it has the width 92 00:04:53,413 --> 00:04:55,330 and height of the file, 93 00:04:56,292 --> 00:05:00,263 and then it has the maximum character value, 94 00:05:00,263 --> 00:05:04,132 which is 255 usually, and a newline. 95 00:05:04,132 --> 00:05:07,194 So we are just skipping this initial part of it, 96 00:05:07,194 --> 00:05:10,242 and then it just has the colors, 97 00:05:10,242 --> 00:05:13,614 where each of the RGB colors is a single character, 98 00:05:13,614 --> 00:05:16,293 in English reading order from left to right, 99 00:05:16,293 --> 00:05:20,376 and I'm just reading that into this wood texture. 100 00:05:21,263 --> 00:05:23,384 In order to do texture mapping, 101 00:05:23,384 --> 00:05:27,773 each vertex must have the texture coordinate. 102 00:05:27,773 --> 00:05:30,891 We saw this earlier, in fact if you remember 103 00:05:30,891 --> 00:05:33,174 when we defined the floor, 104 00:05:33,174 --> 00:05:36,112 that we had the floor, and we had texture coordinates 105 00:05:36,112 --> 00:05:39,624 from the floor, ranging from (0,0) to (1,1). 106 00:05:39,624 --> 00:05:42,094 But each vertex in the general case, 107 00:05:42,094 --> 00:05:45,410 must have a texture coordinate associated with it. 108 00:05:45,410 --> 00:05:49,745 When you interpolate or rasterize the vertices in OpenGL, 109 00:05:49,745 --> 00:05:51,978 it interpolates these texture coordinates 110 00:05:51,978 --> 00:05:54,622 to get a texture coordinate for each pixel, 111 00:05:54,622 --> 00:05:58,546 which will then be used to look up into the texture. 112 00:05:58,546 --> 00:06:00,276 So the first part of this is setting up 113 00:06:00,276 --> 00:06:02,076 the texture coordinates, 114 00:06:02,076 --> 00:06:06,243 so your glGenTextures just sets up one buffer for texNames. 115 00:06:07,915 --> 00:06:10,251 You bind the Vertex Array corresponding to 116 00:06:10,251 --> 00:06:13,433 the vertex array objects for the floor, 117 00:06:13,433 --> 00:06:17,100 you bind the buffer, and this will be the buffers 118 00:06:18,932 --> 00:06:21,765 numobjects*numperobj+ncolors, 119 00:06:22,733 --> 00:06:25,161 so you're adding a buffer corresponding 120 00:06:25,161 --> 00:06:27,411 to the texture information. 121 00:06:28,451 --> 00:06:32,041 And the information of the buffer detail there, 122 00:06:32,041 --> 00:06:34,374 corresponds to the floortex, 123 00:06:35,263 --> 00:06:39,144 which is the information regarding the floor texture. 124 00:06:39,144 --> 00:06:41,852 The buffer data is stored in the same way 125 00:06:41,852 --> 00:06:45,135 as you've done vertices and colors. 126 00:06:45,135 --> 00:06:48,697 Remember that we used layout 0 for the vertices, 127 00:06:48,697 --> 00:06:51,197 and layout 1 for the colors. 128 00:06:52,428 --> 00:06:56,979 We are now going to use layout location 2 for the texture coordinatess. 129 00:06:56,979 --> 00:07:01,018 So you enable the VertexAttribArray(2). 130 00:07:01,018 --> 00:07:03,685 You set the VertexAttribPointer, 131 00:07:04,598 --> 00:07:08,960 and now notice that you have only two elements, 132 00:07:08,960 --> 00:07:13,540 which are the S and T coordinates for the texture. 133 00:07:13,540 --> 00:07:16,050 You also need to define an ActiveTexture, 134 00:07:16,050 --> 00:07:18,488 which is texture 0 in this case. 135 00:07:18,488 --> 00:07:20,957 You need to enable texturing, 136 00:07:20,957 --> 00:07:24,280 and you need to bind to GL_TEXTURE_2D, 137 00:07:24,280 --> 00:07:25,613 the texNames[0]. 138 00:07:26,527 --> 00:07:30,694 All of this setup is needed to be able to use the textures. 139 00:07:32,522 --> 00:07:37,000 The next part of this is Specifying the Texture Image. 140 00:07:37,000 --> 00:07:39,880 And this is a command glTexImage2D, 141 00:07:39,880 --> 00:07:42,881 which takes in a number of different arguments. 142 00:07:42,881 --> 00:07:45,821 The target is just GL_TEXTURE_2D. 143 00:07:45,821 --> 00:07:48,429 In principle you could have more interesting versions 144 00:07:48,429 --> 00:07:50,660 like TEXTURE_3D and 4D, 145 00:07:50,660 --> 00:07:54,179 but we're just sticking with standard TEXTURE_2D. 146 00:07:54,179 --> 00:07:56,630 Level is relevant for mip-mapping, 147 00:07:56,630 --> 00:07:59,199 which we are not really getting to in this course, 148 00:07:59,199 --> 00:08:01,663 so it's almost always going to be zero. 149 00:08:01,663 --> 00:08:04,461 The number of components is three for RGB, 150 00:08:04,461 --> 00:08:06,791 it would be four for RGBA. 151 00:08:06,791 --> 00:08:09,311 Width and height must be a power of 2, 152 00:08:09,311 --> 00:08:11,973 if you want things to work more simply. 153 00:08:11,973 --> 00:08:14,473 Border is usually set to zero. 154 00:08:15,592 --> 00:08:19,138 The format will be GL_RGB or GL_RGBA, 155 00:08:19,138 --> 00:08:22,362 and then the type is what is the type of input data. 156 00:08:22,362 --> 00:08:24,334 In our case it's GL_UNSIGNED_BYTE, 157 00:08:24,334 --> 00:08:28,362 but it could also be GL_FLOAT, etc. 158 00:08:28,362 --> 00:08:31,445 So we define the glTexImage2D command 159 00:08:33,991 --> 00:08:37,143 with GL_TEXTURE_2D, 0, GL_RGB, 160 00:08:37,143 --> 00:08:40,961 the width and height of our texture is 256, 256, 161 00:08:40,961 --> 00:08:43,163 GL_RGB, GL_UNSIGNED_BYTE, 162 00:08:43,163 --> 00:08:45,073 and then you give the texture input, 163 00:08:45,073 --> 00:08:46,906 which is woodtexture. 164 00:08:47,869 --> 00:08:50,091 The following lines or parameters 165 00:08:50,091 --> 00:08:53,641 will say how the texture map is accessed, 166 00:08:53,641 --> 00:08:56,761 which is essentially that you have a texture map 167 00:08:56,761 --> 00:08:58,572 at a certain resolution. 168 00:08:58,572 --> 00:09:01,861 But when you map it onto the object geometry, 169 00:09:01,861 --> 00:09:04,218 you will end up either magnifying 170 00:09:04,218 --> 00:09:06,363 or expanding the texture map, 171 00:09:06,363 --> 00:09:10,371 or minifying or reducing the size of the texture map. 172 00:09:10,371 --> 00:09:13,392 The question is, how do you handle this? 173 00:09:13,392 --> 00:09:16,860 And do you do linear interpolation for magnification, 174 00:09:16,860 --> 00:09:19,453 or do you just take nearest point? 175 00:09:19,453 --> 00:09:22,190 And in this case we've specified GL_LINEAR 176 00:09:22,190 --> 00:09:25,832 for both the magnification and minimication filters. 177 00:09:25,832 --> 00:09:28,119 Now one could also use mipmaps, 178 00:09:28,119 --> 00:09:30,065 which are texture maps at a variety 179 00:09:30,065 --> 00:09:32,976 of different resolutions from the finest resolution 180 00:09:32,976 --> 00:09:35,314 to the coarsest resolution. 181 00:09:35,314 --> 00:09:38,422 And in this case you would specify GL_MIPMAP_LINEAR, 182 00:09:38,422 --> 00:09:40,965 which does bilinear filtering on the texture map, 183 00:09:40,965 --> 00:09:43,701 but also linear filtering of mipmap levels, 184 00:09:43,701 --> 00:09:48,032 and that is the most expensive trilinear filtering. 185 00:09:48,032 --> 00:09:49,948 Finally, we have these two parameters, 186 00:09:49,948 --> 00:09:52,441 which is what happens if you go beyond 187 00:09:52,441 --> 00:09:54,990 the range of 0 to 1 in S and T. 188 00:09:54,990 --> 00:09:57,261 Do you clamp to 1 and 0, 189 00:09:57,261 --> 00:10:00,095 or do you repeat, and do you loop around, 190 00:10:00,095 --> 00:10:02,345 then 1.5 comes back to 0.5? 191 00:10:05,301 --> 00:10:07,949 In order to use texture maps in the shader, 192 00:10:07,949 --> 00:10:10,282 we need to define a sampler. 193 00:10:10,282 --> 00:10:13,161 That's what GLint texsampler is doing. 194 00:10:13,161 --> 00:10:16,000 It is getting the location from the program 195 00:10:16,000 --> 00:10:17,660 of this tex variable. 196 00:10:17,660 --> 00:10:19,762 You initially set it to zero, 197 00:10:19,762 --> 00:10:22,509 which could also be the value of GL_TEXTURE0, 198 00:10:22,509 --> 00:10:25,245 so it's saying which texture map if we have several, 199 00:10:25,245 --> 00:10:27,011 should I look up? 200 00:10:27,011 --> 00:10:30,881 And then we set istex to the glGetUniformLocation 201 00:10:30,881 --> 00:10:33,221 of the istex command in the program, 202 00:10:33,221 --> 00:10:36,138 and that will be set appropriately. 203 00:10:38,341 --> 00:10:41,681 Let's now talk about drawing with texture. 204 00:10:41,681 --> 00:10:44,082 We add a function, draw with texture 205 00:10:44,082 --> 00:10:45,861 which will be used for the floor, 206 00:10:45,861 --> 00:10:48,992 which is similar to draw object, 207 00:10:48,992 --> 00:10:52,659 except now it takes as an input, the texture. 208 00:10:52,659 --> 00:10:56,159 So we first bind the TEXTURE_2D to the texture. 209 00:10:57,731 --> 00:11:00,762 You glBindVertexArray for the object, 210 00:11:00,762 --> 00:11:02,582 and now you just draw it. 211 00:11:02,582 --> 00:11:04,672 So the only thing that's different is 212 00:11:04,672 --> 00:11:07,020 that you're binding this texture 213 00:11:07,020 --> 00:11:09,902 corresponding to the texture variable. 214 00:11:09,902 --> 00:11:12,221 Here are the final steps for drawing. 215 00:11:12,221 --> 00:11:15,330 The vertex shader is just passing on the texture coordinates 216 00:11:15,330 --> 00:11:16,928 to the fragment shader. 217 00:11:16,928 --> 00:11:20,168 Notice that layout location 2, 218 00:11:20,168 --> 00:11:22,322 is where the texture coordinates come in. 219 00:11:22,322 --> 00:11:25,898 Those are a vec2, they're just S and T coordinates. 220 00:11:25,898 --> 00:11:28,742 The output from this will be the texture coordinates 221 00:11:28,742 --> 00:11:30,760 going into the shader, 222 00:11:30,760 --> 00:11:34,051 fragment shader, just like positions and normals. 223 00:11:34,051 --> 00:11:35,698 There is this istex. 224 00:11:35,698 --> 00:11:37,428 And now let's look at the main() routine. 225 00:11:37,428 --> 00:11:39,360 You're doing projection * modelview 226 00:11:39,360 --> 00:11:41,701 as before. You set mynormal and myvertex 227 00:11:41,701 --> 00:11:43,670 for the fragment shader. 228 00:11:43,670 --> 00:11:46,192 You set texcoord initially to (0,0), 229 00:11:46,192 --> 00:11:49,210 which is a default value to prevent errors, 230 00:11:49,210 --> 00:11:51,548 and if texturing is turned on, 231 00:11:51,548 --> 00:11:53,630 if istex not equal to zero, 232 00:11:53,630 --> 00:11:57,161 then you simply set it to the input texture coordinates 233 00:11:57,161 --> 00:11:58,269 that you received. 234 00:11:58,269 --> 00:12:00,490 So it's just passing on the texture coordinates 235 00:12:00,490 --> 00:12:02,075 to the fragment shader. 236 00:12:02,075 --> 00:12:04,780 Notice that the OpenGL rasterization hardware 237 00:12:04,780 --> 00:12:06,739 will interpolate texture coordinates 238 00:12:06,739 --> 00:12:09,801 between the vertices and so each pixel 239 00:12:09,801 --> 00:12:13,261 will get the correct interpolated texture coordinate. 240 00:12:13,261 --> 00:12:16,875 The fragment shader in this case is very simple, 241 00:12:16,875 --> 00:12:20,241 although it can be a more complex blend. 242 00:12:20,241 --> 00:12:24,408 So it takes this uniform 2D sampler tex, this istex command, 243 00:12:26,380 --> 00:12:31,010 and it simply says, if (istex > 0) then the 244 00:12:31,010 --> 00:12:35,021 fragColor will be given by this texture 245 00:12:35,021 --> 00:12:37,392 of (tex, texcoord). 246 00:12:37,392 --> 00:12:39,578 That is the relevant part of the fragment shader, 247 00:12:39,578 --> 00:12:41,981 for texturing. Of course, it will also do 248 00:12:41,981 --> 00:12:44,811 other commands for lighting if we are not texturing. 249 00:12:44,811 --> 00:12:46,982 And of course this can be a more complicated blend; 250 00:12:46,982 --> 00:12:49,329 you could do the standard lighting calculation, 251 00:12:49,329 --> 00:12:51,002 modulate with the texture, 252 00:12:51,002 --> 00:12:53,071 but in this case we are just looking up 253 00:12:53,071 --> 00:12:55,858 and using the texture map directly. 254 00:12:55,858 --> 00:12:57,748 Going back to our program, 255 00:12:57,748 --> 00:12:59,782 I can toggle between the texture, 256 00:12:59,782 --> 00:13:02,058 no texturing and texturing. 257 00:13:02,058 --> 00:13:04,290 I can move back and forth, 258 00:13:04,290 --> 00:13:06,391 and I have my scene now, 259 00:13:06,391 --> 00:13:10,308 with my floor, pillars, teapots, with texturing.