Games101

Games101学习

Transformation(变换)

What is transformation？
- 类比照相
  - model trans(摆物体)
  - view trans(找角度)(改变的是相机)
  - projection trans(投影，变成照片)

View Transformation/Model

Define the camera
- position
- gaze direction
- up direction
camera：The origin；up at Y；look at -Z
Transform camera by M~view~
- Trans e to origin
- Rotate g to -Z
- Rotate t to Y
- Rotate (g x t) to X
$M_{view} = R_{view}T_{view}$
Trans e to origin
- $T_{view} = \left[\begin{matrix} 1 & 0 & 0 & -x_e\\ 0 & 1 & 0 & -y_e\\ 0 & 0 & 1 & -z_e\\ 0 & 0 & 0 & 1\\ \end{matrix}\right]$

Rotate R~view~
- Rotate g to -Z … is hard
- inverse is easy eg: X to(g x t)

$R_{view}^{-1} = \left[\begin{matrix} x_{gxt} & x_t & x_{-g} & 0\\ y_{gxt} & y_t & y_{-g} & 0\\ z_{gxt} & z_t & z_{-g} & 0\\ 0 & 0 & 0 & 1\\ \end{matrix}\right]$

旋转矩阵是正交矩阵
- $R_{view} = \left[\begin{matrix} x_{gxt} & y_{gxt} & z_{gxt} & 0\\ x_t & y_t & z_t & 0\\ x_{-g} & y_{-g} & z_{-g} & 0\\ 0 & 0 & 0 & 1\\ \end{matrix}\right]$

Projection Transformation

Perspective projection vs. orthographic projection

Orthographic Projection(正交)

simple understand
- camera pos (at origin; look at -Z; up at Y)
- Drop Z coordinate
- Translate and scale [-1,1]
general
- a cuboid [l,r] x [b,t] x [f,n]
  - 沿-Z方向看，远小于近(n > f)
- map to cannonical cube [-1,1]^3^
  - translate
  - scale
- Transform matrix
  - $R_{ortho} = \left[\begin{matrix} \frac{2}{r - l} & 0 & 0 & 0\\ 0 & \frac{2}{t - b} & 0 & 0\\ 0 & 0 & \frac{2}{n - f} & 0\\ 0 & 0 & 0 & 1\\ \end{matrix}\right] \left[\begin{matrix} 1 & 0 & 0 & -\frac{r + l}{2}\\ 0 & 1 & 0 & -\frac{t + b}{2}\\ 0 & 0 & 1 & -\frac{n + f}{2}\\ 0 & 0 & 0 & 1\\ \end{matrix}\right]$

Perspective Projection(透视)

近大远小
平行线不再相交
(x, y, z, 1) 与 (kx, ky, kz, k) 在空间中表示的是同一个点
How to do perspective projection
- First squish the frustum(中文：视锥) to cubiod (n -> n; f -> f)==M~persp->ortho~==
- Do orthographic projection ==M~ortho~==(already konw)
侧视图
- $y’ = \frac{n}{z}y\\ x' = \frac{n}{z}x$
- $\left[\begin{matrix} x\\ y\\ z\\ 1\\ \end{matrix}\right] \Longrightarrow \left[\begin{matrix} \frac{nx}{z}\\ \frac{nx}{y}\\ unkonwn\\ 1\\ \end{matrix}\right] \stackrel{\mathrm{minus\; z}}{==} \left[\begin{matrix} nx\\ ny\\ still\; unkown\\ z\\ \end{matrix}\right]$
- so
  $M_{persp->ortho}^{(4\; x\; 4)} \left[\begin{matrix} x\\ y\\ z\\ 1\\ \end{matrix}\right] = \left[\begin{matrix} nx\\ ny\\ still\; unkown\\ z\\ \end{matrix}\right]$
- figure out
  $M_{persp->ortho} = \left[\begin{matrix} n & 0 & 0 & 0\\ 0 & n & 0 & 0\\ ? & ? & ? & ?\\ 0 & 0 & 1 & 0\\ \end{matrix}\right]$
- $M_{persp->ortho}^{(4\; x\; 4)} \left[\begin{matrix} x\\ y\\ z\\ 1\\ \end{matrix}\right] = \left[\begin{matrix} nx\\ ny\\ still\; unkown\\ z\\ \end{matrix}\right]$
  结果最后一个值需要是z
- 近平面的点不会发生任何变化
  - replace z with n (keep z)
  - $\left[\begin{matrix} x\\ y\\ n\\ 1\\ \end{matrix}\right] \Longrightarrow \left[\begin{matrix} x\\ y\\ n\\ 1\\ \end{matrix}\right] \stackrel{}{==} \left[\begin{matrix} nx\\ ny\\ n^2\\ n\\ \end{matrix}\right]$
    → 指代变换
  - n^2^与 x 和 y 无关，有：
  - $\left(\begin{matrix} 0 & 0 & A & B \end{matrix}\right) \left(\begin{matrix} x\\ y\\ n\\ 1\\ \end{matrix}\right) =\; n^2$
  - $An + B = n^2$

远平面的点Z不会发生任何变化
- 远平面特殊点：中心点
- keep z
- $\left[\begin{matrix} 0\\ 0\\ f\\ 1\\ \end{matrix}\right] \Longrightarrow \left[\begin{matrix} 0\\ 0\\ f\\ 1\\ \end{matrix}\right] \stackrel{}{==} \left[\begin{matrix} 0\\ 0\\ f^2\\ f\\ \end{matrix}\right]$
- $Af + B = f^2$
Solve A and B
- $An + B = n^2\qquad A = n + f\\ Af + B = f^2\qquad\space B = -nf$

M~persp->ortho~ has solved !
$M_{persp->ortho} = \left[\begin{matrix} n & 0 & 0 & 0\\ 0 & n & 0 & 0\\ 0 & 0 & n+f & -nf\\ 0 & 0 & 1 & 0\\ \end{matrix}\right]$

Next :
- Do orthographic projection
- $M_{persp} = M_{ortho}M_{persp->ortho}$

Rasterization(光栅化)

fov：field of view; 常用垂直可视角度fovY
$aspect\ ratio = width / height$

Cannonical Cube to Screen

What is Screen？
- an array of pixels
- size of array: resolution
- a typical kind of raster display
raster == screen
- rasterize == drawing onto screen
pixel(picture element)
color is a mixture of (red,green,blue)
Screen space
- 像素坐标是整数
- from (0 , 0) to (width - 1 , height - 1)
- 像素中心:(x + 0.5 , y + 0.5)
- 屏幕覆盖(0 , 0) to (width , height)
What to do
- irrelevant to z
- transform in xy plane: [-1 , 1]^2^ to [0 , width] x [0 , height]
- viewport transform matrix:
  - $M_{viewport} = \left(\begin{matrix} \frac{width}{2} & 0 & 0 & \frac{width}{2}\\ 0 & \frac{height}{2} & 0 & \frac{height}{2}\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ \end{matrix}\right)$

Drawing on Raster Displays

Triangles - Fundamental Shape Primitives
- most basic polygon
- 三角形内部一定是平面
- 三角形内外定义确定
- 内部点的属性值可以从三个顶点插值计算
- 注：向量叉积可以确定点是否在三角形内部
光栅化重要概念:判断像素(中心点)和三角形的位置关系

Sample(采样)

什么是采样：
- 回答连续函数在不同位置的值
- 把函数离散化
Sampling is a core idea in graphics

2D Sampling

sample if each pixel center is inside triangle

Define Binary Func:
1
inside(tri, x, y)
$inside(t,\ x,\ y) = \begin{cases} 1, & \text{Point (x , y) in triangle t} \\ 0, & \text{otherwise} \\ \end{cases}$

Rasterization = Sampling a 2D Indicator Func

1
2
3

for(int x = 0; x < xmax; ++x)
	for(int y = 0; y < ymax; ++y)
		image[x][y] = inside(tri, x + 0.5, y + 0.5)

evaluate inside

叉乘
边界点要么不做处理要么特殊处理
Checking All Pixels on Screen:
- Use Bounding Box(AABB)
- 避免屏幕所有的点都进行运算，只计算包围盒内的点
- (加速处理)
Jaggies!(锯齿)
- 产生原因：采样率不够
Aliasing(Jaggies) 抗锯齿/反走样

Antialiasing and Z-Buffering(反走样和深度缓冲)

采样可以发生在不同的位置、不同的时间

Sampling Artifacts

Jaggies
Moire Patterns
and more…
Why?
- changing too fast(high frequency)
- sampling too slowly

Pre-Filtering Before Sampling(模糊操作)

Blurred Aliasing

Sample then filter(Wrong!)

Aliasing

傅里叶变换
频率分析定义：同样的采样方法，采样两种不同频率的函数，结果无法区分。
采样图片：中间频率低，亮度表示信息量
- High-pass Filter(高通滤波)
  - 显示图像边界(发生剧烈变化)
- Low-pass Filter
  - 图像细节消失

Convolution(卷积)

Filtering = Convolution = Averaging
在原始信号做平均
时域上的卷积 = 频域上的乘积

Option 1

卷积的滤波器

Option 2

傅里叶变换到频域
卷积滤波器变换到频域
相乘得到频域结果，再逆傅里叶变换到时域

Box Filter

可以看作低通滤波
Wider Filter Kernel = Lower Frequency(越大的box越模糊)

Sampling = Repeating Frequency Contents

采样中时域和频域相反，采样间隔越长，对应频域越密集，产生走样

How to reduce Aliasing Error?

先去除高频信息，再采样

A Practical Pre-Filter

1 pixel-width box filter(low pass,blurring)

Solution

Convolve f(x,y) by 1-pixel box-blur
- convolving = filtering = averaging
Sample at every pixel center

Antialiasing By SuperSampling(MSAA)

在一pixel内部增加更多的采样点
blur result:
MSAA解决的是对信号的模糊操作
但是计算量增大，实际应用中采样点复用

其余抗锯齿方法

FXAA(Fast Approximate AA)

与采样无关
找到边界，更换无锯齿边界

TAA(Temporal AA)

复用上一帧感知的信息

超分辨率

与超采样相同，都要解决样本量不足的问题
DLSS(Deep Learning Super Sampling)

Z-buffer

Painter’s Algorithm
- overwrite in the framebuffer(前面的覆盖后面的)
- requires sorting in the depth

z-buffer algorithm

store min z-value for each sample
frame buffer(结果) stores color values
depth buffer(深度) stores depth

```c++
Initailize depth buffer to ∞
During rasterization:
for(each triangle T)

for(each sample(x,y,z) in T)
    if(z < zbuffer[x,y])
        framebuffer[x,y] = rgb;          //update color
        zbuffer[x,y] = z;                //update depth
    else
        ;    //do mothing


  - <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923105927171.png" alt="image-20240923105927171" style="zoom:50%;" />

## Shading

- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923112117973.png" alt="image-20240923112117973" style="zoom: 50%;" />
- shading: darkening or coloring(明暗与颜色)
- applying a material to an object(对不同的物体应用不同的材质)

### Blinn-Phong Reflectance Model(simple shading model)

- specular highlights(高光/镜面反射)
- diffuse reflection(漫反射)
- ambient lighting(间接光照)
- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923114019524.png" alt="image-20240923114019524" style="zoom:50%;" />
- Inputs:
  - Viewer direction, v
  - Surface normal, n
  - Light direction, l
  - (表示方向应该是单位向量，长度永远是1)
  - Surface parameters(color,shininess,...)
- **Shading is Local**(shading != shadow)

#### Diffuse Reflection

- Shding independent of view direction

  - $$
    L_{d} = k_{d}\ (I/r^{2})\ max(0,n\cdot l)
    $$

  - $L_{d}$: diffusely refelcted light

  - $k_{d}$: diffuse coefficient(color)(1:表示不吸收，全部反射出去，亮；0:表示全部吸收，黑色)

  - $(I/r^{2})$: energy arrived at the shding point

  - $max(0,n\cdot l)$: energy received by shading point(被吸收)

  - 漫反射与v无关，各个方向上应该相同

  - <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923120901266.png" alt="image-20240923120901266" style="zoom:50%;" />

#### Specular Term

- Bright near mirror reflection direction

- v 接近 r 的时候，可以看到高光

  - v close to mirror direction == half vector near normal (计算r是困难的)

- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923124036831.png" alt="image-20240923124036831" style="zoom:50%;" />

- $$
  \begin{align*}
  h(半程向量) &= bisector(v,l)\\ 
             &= \frac{v+l}{||v+l||}
  \end{align*}
  $$

- $$
  \begin{align*}
  L_{s} &= k_{s}\ (I/r^{2})\ max(0,cos\alpha)^{p}\\
        &= k_{s}\ (I/r^{2})\ max(0,n\cdot h)^{p}
  \end{align*}
  $$

  - $k_{s}$: specular coefficient, 通常认为是白色(亮度)
  - $L_{s}$: specularly reflected light
  - p: narrows reflection lobe (控制高光大小) (100-200)
  - <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923125723045.png" alt="image-20240923125723045" style="zoom:33%;" />

#### Ambient Term

- $$
  L_{a} = k_{a}\ I_{a}
  $$

  - $I_{a}$: 每个点的环境光强度是相同的
  - $K_{a}$: ambient coefficient
  - $L_{a}$: reflected ambient light
  - 此处的环境光可以看作常数

#### Result

- Bling-Phong Reflection =  Ambient + Diffuse + Specular 

- $$
  \begin{align*}
  L &= L_{a} + L_{d} + L_{s}\\
    &= k_{a}\ I_{a} + k_{d}\ (I/r^{2})\ max(0,n\cdot l) + k_{s}\ (I/r^{2})\ max(0,n\cdot h)^{p}
  \end{align*}
  $$

### Shading Frequencies

- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923154335797.png" alt="image-20240923154335797" style="zoom:33%;" />
- Shade each triangle(flat shding)
- Shade each vertex(gouraud shading)
- Shade each pixel(Phong sahding)

#### Flat shading

- triangle face is flat- one normal factor
- not ggod for smooth surfaces

#### Gouraud shading

- Interpolate colors from vertices across triangle
- each vertex has a normal vector

#### Phong shading 

- Interpolate normal vectors across each triangle
- Compute full shading model at each pixel

#### Defining Per-Vertex Normal Vectores

- Simple scheme: average surrounding face normals

- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923160356734.png" alt="image-20240923160356734" style="zoom:50%;" />

- $$
  N_{v} = \frac{\sum_{i}{N_{i}}}{||\sum_{i}{N_{i}}||}
  $$

#### Defining Per-Pixel Normal Vectors

- Barycnetric interpolation of vertex normals(normalize)
- 

### Graphics(Real-time Rendering) Pipeline

- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923161609029.png" alt="image-20240923161609029" style="zoom:50%;" />
- 从三维场景到2D图像

### Texture Mapping

- surfaces are 2D
- <img src="https://qitiantaile.oss-cn-guangzhou.aliyuncs.com/blog/image-20240923163758865.png" alt="image-20240923163758865" style="zoom:50%;" />

#### Visualization of Texture Coordinates

- each triangle vertex is assigned a texture coordinate(u,v)
- 通常认为uv范围在[0,1]

```c++
for each rasterized screen sample(x,y):           //usually a pixel's center
	(u,v) = evaluate texture coordinate at (x,y)  //Using barycentric coordinates
	texcolor = texture.sample(u,v);
	set sample's color to texcolor;

Texture Magnification(texture is too small)

a pixel on a texture ————-a texel(纹理像素)

Bilinear interpolation(双线性插值)

Linear interpolation
- $lerp(x,v_{0},v_{1}) = v_{0}+x(v_{1}-v_{0})$
- $\begin{align*} u_{0} &= lerp(s,u_{00},u_{10})\\ u_{1} &= lerp(s,u_{01},u_{11}) \end{align*}$
- $f(x,y) = lerp(t,u_{0},u_{1})$

Texture Magnification(texture is too big)

What if don’t sample
get the average value at once
实质是点查询和范围查询问题

Mipmap

fast, approx, square
只多了1/3的存储量

Computing Mipmap Level D

区域可以近似为正方形区域

Trilinear Interpolation

计算离散的mipmap层

Anisotropic Flitering(各向异性过滤)

也叫做RipMap
可以进行矩形的查询

Interpolation Across Triangles(插值)

Specify values at vertices
smoothly varying values across trangles
interpolate what?
- texture coordinates,colors normal vectors,…
How to interpolate

Barycentric Coordinates

属性混合
- $V = \alpha V_{A}+\beta V_{B} + \gamma V_{C}$
- $V_{A}$… can be positions,texture coordinates,color,normal,depth,materical attributes
- 投影后的重心坐标可能会发生改变