Panoramas Pt.2 – Gestures and Rounded Corners

Jump to: Gesture Controls | Navigation Buttons | Rounded Corners

Abstract

This second article on Splash Reader’s Panorama mode details the next evolution of the feature: rounded corners for visual polish, gesture-based interactions for scaling, dragging, and rotating, and navigation button attachments. The goal is to more closely match Apple’s Panorama feel while improving immersion and control during multi-page reading.

Read the first article in this series here.

Download Splash Reader on the App Store

Problems with previous versions

• Placement and orientation were rigid. Repositioning required a crown long press and rotation was constrained to the Y-axis, preventing setups like ceiling placement for reading while lying down. Scaling was not possible.
• Navigation via a separate remote felt cumbersome and occasionally non-intuitive.
• Sharp corners reduced immersion and diverged from the rounded presentation of Apple’s Panorama.

Gesture Controls

I use GitHub search frequently when looking for other developers solutions for problems I face. It proved useful for identifying possible solutions for complex gesture interactions.

An extension pattern for RealityView (often named installGestures) provides drag, scale, and rotation out of the box. The underlying origin is unclear, but the approach is widely adopted and easy to adapt.

For Splash Reader, the gesture logic was adjusted and BillboardComponent was used to keep the panorama facing the user. In practice, this yielded smoother and more predictable behavior than the extension’s built-in facing logic.

public extension RealityView {
  /// Apply this to a \RealityView\ to pass gestures on to the component code.
  func installGestures() -> some View {
    simultaneousGesture(dragGesture)
      .simultaneousGesture(magnifyGesture)
      .simultaneousGesture(rotateGesture)
  }

  /// Builds a drag gesture.
  var dragGesture: some Gesture {
    DragGesture()
      .targetedToAnyEntity()
      .useGestureComponent()
  }

  /// Builds a magnify gesture.
  var magnifyGesture: some Gesture {
    MagnifyGesture()
      .targetedToAnyEntity()
      .useGestureComponent()
  }

  /// Builds a rotate gesture.
  var rotateGesture: some Gesture {
    RotateGesture3D()
      .targetedToAnyEntity()
      .useGestureComponent()
  }
}

// MARK: - Rotate -
public extension Gesture where Value == EntityTargetValue<RotateGesture3D.Value> {
  /// Connects the gesture input to the 'GestureComponent' code.
  func useGestureComponent() -> some Gesture {
    onChanged { value in
      guard var gestureComponent = value.entity.gestureComponent else { return }
      gestureComponent.onChanged(value: value)
      value.entity.components.set(gestureComponent)
    }
    .onEnded { value in
      guard var gestureComponent = value.entity.gestureComponent else { return }
      gestureComponent.onEnded(value: value)
      value.entity.components.set(gestureComponent)
    }
  }
}

// MARK: - Drag -
public extension Gesture where Value == EntityTargetValue<DragGesture.Value> {
  /// Connects the gesture input to the 'GestureComponent' code.
  func useGestureComponent(completion: @escaping (_ChangedGesture<Self>.Value) -> Void = {_ in }) -> some Gesture {
    onChanged { value in
      guard var gestureComponent = value.entity.gestureComponent else { return }
      gestureComponent.onChanged(value: value)
      value.entity.components.set(gestureComponent)
    }
    .onEnded { value in
      guard var gestureComponent = value.entity.gestureComponent else { return }
      gestureComponent.onEnded(value: value)
      value.entity.components.set(gestureComponent)
      completion(value)
    }
  }
}

// MARK: - Magnify (Scale) -
public extension Gesture where Value == EntityTargetValue<MagnifyGesture.Value> {
  /// Connects the gesture input to the 'GestureComponent' code.
  func useGestureComponent() -> some Gesture {
    onChanged { value in
      guard var gestureComponent = value.entity.gestureComponent else { return }
      gestureComponent.onChanged(value: value)
      value.entity.components.set(gestureComponent)
    }
    .onEnded { value in
      guard var gestureComponent = value.entity.gestureComponent else { return }
      gestureComponent.onEnded(value: value)
      value.entity.components.set(gestureComponent)
    }
  }
}

public extension Entity {
  var gestureComponent: GestureComponent? {
    get { components[GestureComponent.self] }
    set { components[GestureComponent.self] = newValue }
  }

  /// Returns the position of the entity specified in the app's coordinate system. On
  /// iOS and macOS, which don't have a device native coordinate system, scene
  /// space is often referred to as "world space".
  var scenePosition: SIMD3<Float> {
    get { position(relativeTo: nil) }
    set { setPosition(newValue, relativeTo: nil) }
  }

  /// Returns the orientation of the entity specified in the app's coordinate system. On
  /// iOS and macOS, which don't have a device native coordinate system, scene
  /// space is often referred to as "world space".
  var sceneOrientation: simd_quatf {
    get { orientation(relativeTo: nil) }
    set { setOrientation(newValue, relativeTo: nil) }
  }
}

Navigation Buttons

Adding navigation buttons as attachments was simple, though positioning them correctly with variable plane dimensions required careful implementation. Unfortunately, puck-style attachments performed poorly on curved surfaces due to ray-interaction geometry challenges. Below is a prototype puck implementation I developed that proved incompatible with the curved plane:

The final setup uses three simple buttons: Next Page, Previous Page, and Exit Panorama. Custom icons were preferred over Apple’s defaults to better match the app’s aesthetic. Hover highlighting was simple to add.

Attachment(id: "NextPageButton") {
  Button {
    viewModel.navigateToNextPage()
  } label: {
    Image("NavBow")
      .renderingMode(.template)
      .resizable()
      .foregroundColor(.white)
      .frame(width: 14, height: 25)
      .scaleEffect(x: -1, y: 1)
      .padding(.vertical, 40)
      .padding(.horizontal, 160)
  }
  .buttonStyle(PanoNavigationButtonStyle())
}

if let nextButtonAttachment = attachments.entity(for: "NextPageButton") {
  splashEntity!.addChild(nextButtonAttachment)
  nextButtonAttachment.position = [
    BUTTON_OFFSET + (PANO_HEIGHT / (2 * aspectRatio)),
    0,
    0
  ]
  nextButtonAttachment.scale = [PANO_HEIGHT * 4, PANO_HEIGHT * 4, 1]
  nextButtonAttachment.components.set(BillboardComponent())
}

Rounded Corners

A static USDZ asset could not scale cleanly across arbitrary dimensions; stretching would skew edges and produce unattractive corners. The solution was to generate the curved plane dynamically based on the combined image size and apply a programmatic corner mask so the curvature and corners remain visually consistent.

This approach preserves corner quality at any aspect ratio and keeps the presentation aligned with the rounded design language users expect in VisionOS.

/// Creates a new curved plane mesh with the specified values.
/// - Parameters:
///   - width: Width of the output plane
///   - depth: Depth of the output plane
///   - vertices: Vertex count in the x and z axis
///   - curveMagnitude: The magnitude of the curve (height of the sine wave)
///   - cornerRadius: The magnitude of the corner roundness
/// - Returns: A curved plane mesh
private static func generateCurvedPlane(
  width: Float, depth: Float, vertices: (Int, Int), curveMagnitude: Float = 1.0,
  cornerRadius: Float = 0.5 // New parameter to control corner roundness
) throws -> MeshResource {
  var descr = MeshDescriptor()
  var meshPositions: [SIMD3<Float>] = []
  var indices: [UInt32] = []
  var textureMap: [SIMD2<Float>] = []

  // Calculate normalized corner radius (as a fraction of the smallest dimension)
  let normalizedRadius = cornerRadius * min(width, depth) * 0.5

  for x_v in 0..<(vertices.0) {
    let vertexCounts = meshPositions.count
    for y_v in 0..<(vertices.1) {
      let u = Float(x_v) / Float(vertices.0 - 1)
      let v = Float(y_v) / Float(vertices.1 - 1)

      let xPosition = (u - 0.5) * width
      let zPosition = (0.5 - v) * depth

      // Distance from each corner
      let halfWidth = width * 0.5
      let halfDepth = depth * 0.5

      // Calculate y position based on curved plane formula
      let yPosition = -1 * (curveMagnitude - (pow(xPosition, 2)) * curveMagnitude / pow(width / 2, 2))

      meshPositions.append([xPosition, yPosition, zPosition])

      // Map texture coordinates to match rounded corners
      // We want to map UV from 0 to 1, but considering the rounded corners
      textureMap.append([u, v])

      if x_v > 0 && y_v > 0 {
        // Only add triangles if all of their vertices are within the rounded rectangle
        if isPointInRoundedRect(xPosition, zPosition, width, depth, normalizedRadius) &&
           isPointInRoundedRect((Float(x_v-1) / Float(vertices.0 - 1) - 0.5) * width, (0.5 - Float(y_v-1) / Float(vertices.1 - 1)) * depth, width, depth, normalizedRadius) &&
           isPointInRoundedRect((Float(x_v-1) / Float(vertices.0 - 1) - 0.5) * width, (0.5 - Float(y_v) / Float(vertices.1 - 1)) * depth, width, depth, normalizedRadius) &&
           isPointInRoundedRect((Float(x_v) / Float(vertices.0 - 1) - 0.5) * width, (0.5 - Float(y_v-1) / Float(vertices.1 - 1)) * depth, width, depth, normalizedRadius) {
          indices.append(
            contentsOf: [
              vertexCounts - vertices.1, vertexCounts, vertexCounts - vertices.1 + 1,
              vertexCounts - vertices.1 + 1, vertexCounts, vertexCounts + 1
            ].map { UInt32($0 + y_v - 1) })
        }
      }
    }
  }

  descr.primitives = .triangles(indices)
  descr.positions = MeshBuffer(meshPositions)
  descr.textureCoordinates = MeshBuffers.TextureCoordinates(textureMap)
  return try .generate(from: [descr])
}

// Helper function to determine if a point is inside a rounded rectangle
private static func isPointInRoundedRect(_ x: Float, _ z: Float, _ width: Float, _ depth: Float, _ cornerRadius: Float) -> Bool {
  let halfWidth = width * 0.5
  let halfDepth = depth * 0.5

  // Check if point is outside the rectangle
  if abs(x) > halfWidth || abs(z) > halfDepth {
    return false
  }

  // Check if point is in the main rectangle (not in corner regions)
  if abs(x) <= halfWidth - cornerRadius || abs(z) <= halfDepth - cornerRadius {
    return true
  }

  // Check if point is in one of the corners
  let dx = abs(x) - (halfWidth - cornerRadius)
  let dz = abs(z) - (halfDepth - cornerRadius)
  return dx * dx + dz * dz <= cornerRadius * cornerRadius
}

// Add a helper function to apply rounded corners to an image
private func applyRoundedCornerMask(to image: UIImage, cornerRadius: CGFloat) -> (UIImage, UIImage) {
  let format = UIGraphicsImageRendererFormat()
  format.scale = image.scale
  format.opaque = false

  // Original image remains unchanged
  let originalImage = image

  // Create an opacity mask (white inside rounded rect, black outside)
  let opacityMask = UIGraphicsImageRenderer(size: image.size, format: format).image { context in
    let rect = CGRect(origin: .zero, size: image.size)

    // Fill with black (transparent)
    UIColor.black.setFill()
    UIRectFill(rect)

    // Draw white rounded rectangle (opaque)
    UIColor.white.setFill()
    UIBezierPath(roundedRect: rect, cornerRadius: cornerRadius).fill()
  }

  return (originalImage, opacityMask)
}

// Simplified curved plane function without corner calculations
private static func generateSimpleCurvedPlane(
  width: Float, depth: Float, vertices: (Int, Int), curveMagnitude: Float = 1.0
) throws -> MeshResource {
  var descr = MeshDescriptor()
  var meshPositions: [SIMD3<Float>] = []
  var indices: [UInt32] = []
  var textureMap: [SIMD2<Float>] = []

  for x_v in 0..<(vertices.0) {
    let vertexCounts = meshPositions.count
    for y_v in 0..<(vertices.1) {
      let u = Float(x_v) / Float(vertices.0 - 1)
      let v = Float(y_v) / Float(vertices.1 - 1)

      let xPosition = (u - 0.5) * width
      let yPosition = (0.5 - v) * depth

      // Calculate z position based on curved plane formula (pivoting on Y-axis)
      let zPosition = -1 * (curveMagnitude - (pow(xPosition, 2)) * curveMagnitude / pow(width / 2, 2))

      meshPositions.append([xPosition, yPosition, zPosition])
      textureMap.append([u, 1.0 - v]) // Flip U coordinate to reverse texture horizontally

      if x_v > 0 && y_v > 0 {
        // Flip triangle winding order to face the correct direction
        indices.append(contentsOf: [
          vertexCounts - vertices.1, vertexCounts - vertices.1 + 1, vertexCounts,
          vertexCounts - vertices.1 + 1, vertexCounts + 1, vertexCounts
        ].map { UInt32($0 + y_v - 1) })
      }
    }
  }

  descr.primitives = .triangles(indices)
  descr.positions = MeshBuffer(meshPositions)
  descr.textureCoordinates = MeshBuffers.TextureCoordinates(textureMap)
  return try .generate(from: [descr])
}

Summary

• Rounded corners increase visual polish and immersion.
• Gesture controls unlock flexible placement, scale, and orientation.
• Inline navigation buttons streamline reading without a separate remote.

Together, these updates make Panorama mode more natural, interactive, and aligned with VisionOS design, reinforcing how compelling long-form reading can feel on Apple Vision Pro.