Skip to content

Commit ef02359

Browse files
committedOct 10, 2023
feat: fisheye
1 parent bc12fe0 commit ef02359

File tree

4 files changed

+146
-15
lines changed

4 files changed

+146
-15
lines changed
 

‎README.md

+32-6
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,8 @@ The `native` route of the library **does not** export `Html` or `Loader`. The de
160160
<li><a href="#view">View</a></li>
161161
<li><a href="#rendertexture">RenderTexture</a></li>
162162
<li><a href="#rendercubetexture">RenderCubeTexture</a></li>
163-
<li><a href="#mask">Mask</a></li>
163+
<li><a href="#fisheye">Fisheye</a></li>
164+
<li><a href="#mask">Mask</a></li>
164165
<li><a href="#meshportalmaterial">MeshPortalMaterial</a></li>
165166
</ul>
166167
<li><a href="#modifiers">Modifiers</a></li>
@@ -3586,10 +3587,6 @@ type Props = JSX.IntrinsicElements['texture'] & {
35863587

35873588
#### RenderCubeTexture
35883589

3589-
<p>
3590-
<a href="https://codesandbox.io/s/7qytdw"><img width="20%" src="https://codesandbox.io/api/v1/sandboxes/7qytdw/screenshot.png" alt="Demo"/></a>
3591-
</p>
3592-
35933590
This component allows you to render a live scene into a cubetexture which you can then apply to a material, for instance as an environment map (via the envMap property). The contents of it run inside a portal and are separate from the rest of the canvas, therefore you can have events in there, environment maps, etc.
35943591

35953592
```tsx
@@ -3607,7 +3604,7 @@ export type RenderCubeTextureProps = Omit<JSX.IntrinsicElements['texture'], 'rot
36073604
/** Optional frame count, defaults to Infinity. If you set it to 1, it would only render a single frame, etc */
36083605
frames?: number
36093606
/** Optional event compute, defaults to undefined */
3610-
compute?: (event: any, state: any, previous: any) => false | undefined
3607+
compute?: ComputeFunction
36113608
/** Flip cubemap, see https://github.com/mrdoob/three.js/blob/master/src/renderers/WebGLCubeRenderTarget.js */
36123609
flip?: boolean
36133610
/** Cubemap resolution (for each of the 6 takes), null === full screen resolution, default: 896 */
@@ -3641,6 +3638,35 @@ const api = useRef<RenderCubeTextureApi>(null!)
36413638
<mesh />
36423639
```
36433640

3641+
### Fisheye
3642+
3643+
<p>
3644+
<a href="https://codesandbox.io/s/7qytdw"><img width="20%" src="https://codesandbox.io/api/v1/sandboxes/7qytdw/screenshot.png" alt="Demo"/></a>
3645+
</p>
3646+
3647+
```tsx
3648+
export type FisheyeProps = JSX.IntrinsicElements['mesh'] & {
3649+
/** Zoom factor, 0..1, 0 */
3650+
zoom?: number
3651+
/** Number of segments, 64 */
3652+
segments?: number
3653+
/** Cubemap resolution (for each of the 6 takes), null === full screen resolution, default: 896 */
3654+
resolution?: number
3655+
/** Children will be projected into the fisheye */
3656+
children: React.ReactNode
3657+
/** Optional render priority, defaults to 1 */
3658+
renderPriority?: number
3659+
}
3660+
```
3661+
3662+
This component will take over system rendering. It portals its children into a cubemap which is then projected onto a sphere. The sphere is rendered out on the screen, filling it. You can lower the resolution to increase performance. Six renders per frame are necessary to construct a full fisheye view, and since each facet of the cubemap only takes a portion of the screen full resolution is not necessary. You can also reduce the amount of segments (resulting in edgier rounds).
3663+
3664+
```jsx
3665+
<Fisheye>
3666+
<YourScene />
3667+
</Fisheye>
3668+
```
3669+
36443670
#### Mask
36453671

36463672
<p>

‎src/core/Fisheye.tsx

+110
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
/**
2+
* Event compute by Garrett Johnson https://twitter.com/garrettkjohnson
3+
* https://discourse.threejs.org/t/how-to-use-three-raycaster-with-a-sphere-projected-envmap/56803/10
4+
*/
5+
6+
import * as THREE from 'three'
7+
import * as React from 'react'
8+
import { useFrame, useThree } from '@react-three/fiber'
9+
import { RenderCubeTexture, RenderCubeTextureApi } from './RenderCubeTexture'
10+
11+
export type FisheyeProps = JSX.IntrinsicElements['mesh'] & {
12+
/** Zoom factor, 0..1, 0 */
13+
zoom?: number
14+
/** Number of segments, 64 */
15+
segments?: number
16+
/** Cubemap resolution (for each of the 6 takes), null === full screen resolution, default: 896 */
17+
resolution?: number
18+
/** Children will be projected into the fisheye */
19+
children: React.ReactNode
20+
/** Optional render priority, defaults to 1 */
21+
renderPriority?: number
22+
}
23+
24+
export function Fisheye({
25+
renderPriority = 1,
26+
zoom = 0,
27+
segments = 64,
28+
children,
29+
resolution = 896,
30+
...props
31+
}: FisheyeProps) {
32+
const sphere = React.useRef<THREE.Mesh>(null!)
33+
const cubeApi = React.useRef<RenderCubeTextureApi>(null!)
34+
35+
// This isn't more than a simple sphere and a fixed orthographc camera
36+
// pointing at it. A virtual scene is portalled into the environment map
37+
// of its material. The cube-camera filming that scene is being synced to
38+
// the portals default camera with the <UpdateCubeCamera> component.
39+
40+
const { width, height } = useThree((state) => state.size)
41+
const [orthoC] = React.useState(() => new THREE.OrthographicCamera())
42+
43+
React.useLayoutEffect(() => {
44+
orthoC.position.set(0, 0, 100)
45+
orthoC.zoom = 100
46+
orthoC.left = width / -2
47+
orthoC.right = width / 2
48+
orthoC.top = height / 2
49+
orthoC.bottom = height / -2
50+
orthoC.updateProjectionMatrix()
51+
}, [width, height])
52+
53+
const radius = (Math.sqrt(width * width + height * height) / 100) * (0.5 + zoom / 2)
54+
const normal = new THREE.Vector3()
55+
const sph = new THREE.Sphere(new THREE.Vector3(), radius)
56+
const normalMatrix = new THREE.Matrix3()
57+
58+
const compute = React.useCallback((event, state, prev) => {
59+
// Raycast from the render camera to the sphere and get the surface normal
60+
// of the point hit in world space of the sphere scene
61+
// We have to set the raycaster using the orthocam and pointer
62+
// to perform sphere interscetions.
63+
state.pointer.set((event.offsetX / state.size.width) * 2 - 1, -(event.offsetY / state.size.height) * 2 + 1)
64+
state.raycaster.setFromCamera(state.pointer, orthoC)
65+
if (!state.raycaster.ray.intersectSphere(sph, normal)) return
66+
else normal.normalize()
67+
// Get the matrix for transforming normals into world space
68+
normalMatrix.getNormalMatrix(cubeApi.current.camera.matrixWorld)
69+
// Get the ray
70+
cubeApi.current.camera.getWorldPosition(state.raycaster.ray.origin)
71+
state.raycaster.ray.direction.set(0, 0, 1).reflect(normal)
72+
state.raycaster.ray.direction.x *= -1 // flip across X to accommodate the "flip" of the env map
73+
state.raycaster.ray.direction.applyNormalMatrix(normalMatrix).multiplyScalar(-1)
74+
return undefined
75+
}, [])
76+
77+
useFrame((state) => {
78+
// Take over rendering
79+
if (renderPriority) state.gl.render(sphere.current, orthoC)
80+
}, renderPriority)
81+
82+
return (
83+
<>
84+
<mesh ref={sphere} {...props} scale={radius}>
85+
<sphereGeometry args={[1, segments, segments]} />
86+
<meshBasicMaterial>
87+
<RenderCubeTexture compute={compute} attach="envMap" flip resolution={resolution} ref={cubeApi}>
88+
{children}
89+
<UpdateCubeCamera api={cubeApi} />
90+
</RenderCubeTexture>
91+
</meshBasicMaterial>
92+
</mesh>
93+
</>
94+
)
95+
}
96+
97+
function UpdateCubeCamera({ api }: { api: React.MutableRefObject<RenderCubeTextureApi> }) {
98+
const t = new THREE.Vector3()
99+
const r = new THREE.Quaternion()
100+
const s = new THREE.Vector3()
101+
const e = new THREE.Euler(0, Math.PI, 0)
102+
useFrame((state) => {
103+
// Read out the cameras whereabouts, state.camera is the one *within* the portal
104+
state.camera.matrixWorld.decompose(t, r, s)
105+
// Apply its position and rotation, flip the Y axis
106+
api.current.camera.position.copy(t)
107+
api.current.camera.quaternion.setFromEuler(e).premultiply(r)
108+
})
109+
return null
110+
}

‎src/core/RenderCubeTexture.tsx

+3-9
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import * as THREE from 'three'
22
import * as React from 'react'
3-
import { ReactThreeFiber, createPortal, useFrame, useThree } from '@react-three/fiber'
3+
import { ComputeFunction, ReactThreeFiber, createPortal, useFrame, useThree } from '@react-three/fiber'
44
import { ForwardRefComponent } from '../helpers/ts-utils'
55

66
export type RenderCubeTextureProps = Omit<JSX.IntrinsicElements['texture'], 'rotation'> & {
@@ -17,7 +17,7 @@ export type RenderCubeTextureProps = Omit<JSX.IntrinsicElements['texture'], 'rot
1717
/** Optional frame count, defaults to Infinity. If you set it to 1, it would only render a single frame, etc */
1818
frames?: number
1919
/** Optional event compute, defaults to undefined */
20-
compute?: (event: any, state: any, previous: any) => false | undefined
20+
compute?: ComputeFunction
2121
/** Flip cubemap, see https://github.com/mrdoob/three.js/blob/master/src/renderers/WebGLCubeRenderTarget.js */
2222
flip?: boolean
2323
/** Cubemap resolution (for each of the 6 takes), null === full screen resolution, default: 896 */
@@ -88,12 +88,6 @@ export const RenderCubeTexture: ForwardRefComponent<RenderCubeTextureProps, Rend
8888
}, [fbo])
8989

9090
const [vScene] = React.useState(() => new THREE.Scene())
91-
const uvCompute = React.useCallback((event, state, previous) => {
92-
// https://github.com/pmndrs/react-three-fiber/pull/782
93-
// Events trigger outside of canvas when moved, use offsetX/Y by default and allow overrides
94-
state.pointer.set((event.offsetX / state.size.width) * 2 - 1, -(event.offsetY / state.size.height) * 2 + 1)
95-
state.raycaster.setFromCamera(state.pointer, state.camera)
96-
}, [])
9791

9892
React.useImperativeHandle(forwardRef, () => ({ scene: vScene, fbo, camera: camera.current }), [fbo])
9993

@@ -106,7 +100,7 @@ export const RenderCubeTexture: ForwardRefComponent<RenderCubeTextureProps, Rend
106100
<group onPointerOver={() => null} />
107101
</Container>,
108102
vScene,
109-
{ events: { compute: compute || uvCompute, priority: eventPriority } }
103+
{ events: { compute, priority: eventPriority } }
110104
)}
111105
<primitive object={fbo.texture} {...props} />
112106
<cubeCamera

‎src/core/index.ts

+1
Original file line numberDiff line numberDiff line change
@@ -141,4 +141,5 @@ export * from './RenderTexture'
141141
export * from './RenderCubeTexture'
142142
export * from './Mask'
143143
export * from './Hud'
144+
export * from './Fisheye'
144145
export * from './MeshPortalMaterial'

1 commit comments

Comments
 (1)

vercel[bot] commented on Oct 10, 2023

@vercel[bot]
Please sign in to comment.