> For the complete documentation index, see [llms.txt](https://docs.photoroom.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.photoroom.com/integrations/how-to-correct-the-perspective-of-trading-cards-in-an-ios-app.md). # How to correct the perspective of trading cards in an iOS app Photos of flat items such as trading cards, book covers, documents or packaging are rarely taken perfectly straight on. The result is a skewed rectangle on a cluttered background, which looks poor in a listing and is hard to compare with other items. This tutorial walks through **Card Straight**, an open-source demo iPhone app that photographs a trading card, uses the Remove Background API to find the card's edges, straightens it on-device, and optionally produces a blurred, relit background with the Image Editing API. The full source code is available on GitHub: [Photoroom/api-demo-apps → iOS Perspective Correction App](https://github.com/Photoroom/api-demo-apps/tree/main/iOS%20Perspective%20Correction%20App) (MIT license). ### What the app produces From a single photo, the app creates two kinds of results: * **Transparent**: a tight, perspective-corrected cutout of the card, exported as a PNG with transparency. * **Background blur + relight**: the card straightened in place, with the original background kept but blurred and the lighting harmonized, exported as a JPEG. | *(original photo)* | *Transparent finish* | *Background blur + relight finish* | | :-------------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------: | | ![](https://raw.githubusercontent.com/Photoroom/api-demo-apps/main/iOS%20Perspective%20Correction%20App/docs/examples/charizard-original.jpg) | ![](https://raw.githubusercontent.com/Photoroom/api-demo-apps/main/iOS%20Perspective%20Correction%20App/docs/examples/charizard-transparent.png) | ![](https://raw.githubusercontent.com/Photoroom/api-demo-apps/main/iOS%20Perspective%20Correction%20App/docs/examples/charizard-blurred.jpg) | | ![](https://raw.githubusercontent.com/Photoroom/api-demo-apps/main/iOS%20Perspective%20Correction%20App/docs/examples/mew-original.jpg) | ![](https://raw.githubusercontent.com/Photoroom/api-demo-apps/main/iOS%20Perspective%20Correction%20App/docs/examples/mew-transparent.png) | ![](https://raw.githubusercontent.com/Photoroom/api-demo-apps/main/iOS%20Perspective%20Correction%20App/docs/examples/mew-blurred.jpg) | {% hint style="info" %} The interesting part of this app is the division of labour: **Photoroom does the hard perception work** (where is the card in the image) while **the phone does the geometry it is uniquely placed to do**: warping the pixels to a rectangle, and using its own camera calibration to work out the card's true proportions. The API is called once per photo for the transparent finish, and once more only if the user asks for the blurred finish. {% endhint %} ### Running the app You need Xcode, an iPhone running iOS 17 or later (the Simulator works for everything except the camera), and a Photoroom API key. If you don't have one yet, here are [the steps to create yours](/getting-started/introduction.md#how-can-i-get-my-api-key). 1. Clone the repository and open `iOS Perspective Correction App/ios-perspective-correction.xcodeproj` in Xcode. 2. Select the `ios-perspective-correction` scheme and an iPhone destination. On a physical device, pick your development team under *Signing & Capabilities*. 3. Build and run. Tap the gear icon and paste your API key: it is stored in the device Keychain and never bundled in the app or logged. 4. Tap **Take a photo** and point the camera at a single, fully visible card on a contrasting background. Automatic capture fires once the card is stable and sharp; you can also use the shutter, **Choose from library**, or **Try a sample card** (nine bundled photos that work in the Simulator). 5. Review the result, adjust **Proportions** or **Rotate** if needed, pick the finish in **Settings → Finish**, then save or share. {% hint style="info" %} Use Sandbox mode while you explore: prepend `sandbox_` to your API key and calls are free (results are watermarked). {% endhint %} ### How it works The pipeline has four stages. Two of them call the Photoroom API; the other two run entirely on the device. #### Step 1: Capture and normalize the photo `CameraCapture.swift` does more than take a picture: it configures the session so that each frame comes with a trustworthy **camera intrinsic matrix** (focal length and optical centre, in pixels). It picks the rear physical wide-angle camera, prefers the 4K preset and falls back to 1080p, locks the zoom factor to 1, turns off video stabilization (which crops and shifts the frame), enables geometric distortion correction where supported, and turns on `isCameraIntrinsicMatrixDeliveryEnabled` on the connection. The captured still is the exact video sample buffer that passed the auto-capture checks, read together with its `kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix` attachment, rather than a separate full-resolution photo whose intrinsics could differ. ```swift session.sessionPreset = session.canSetSessionPreset(.hd4K3840x2160) ? .hd4K3840x2160 : .hd1920x1080 device.videoZoomFactor = 1 if connection.isVideoStabilizationSupported { connection.preferredVideoStabilizationMode = .off } if connection.isCameraIntrinsicMatrixDeliverySupported { connection.isCameraIntrinsicMatrixDeliveryEnabled = true } // Later, for the captured frame: let intrinsics = CMGetAttachment(sampleBuffer, key: kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, attachmentModeOut: nil) as? Data ``` Orientation is handled with `AVCaptureDevice.RotationCoordinator`: sensor pixels are left unrotated, the closest cardinal EXIF orientation is attached to the JPEG, and the same orientation is stored alongside the calibration. This matters in Step 3, where the app has to undo orientation and resizing before it can project detected corners into calibrated camera coordinates; otherwise it would pair a portrait, cropped image with a landscape, full-frame matrix. Before anything is sent to Photoroom, `CardProcessor.swift` applies the EXIF orientation and downsizes the photo to at most 3,000 pixels on its longest edge: ```swift let original = CGImageSourceCreateThumbnailAtIndex(source, 0, [ kCGImageSourceCreateThumbnailFromImageAlways: true, kCGImageSourceCreateThumbnailWithTransform: true, // bake in EXIF rotation kCGImageSourceThumbnailMaxPixelSize: 3000 ] as CFDictionary) ``` Both steps matter for API integrations in general: the Remove Background API doesn't carry EXIF metadata into its result, so an un-rotated upload comes back rotated (see Troubleshooting), and a smaller upload means a faster round trip with no visible loss for this use case. The shutter can also fire on its own. An on-device Vision rectangle detector checks preview frames at about 5 Hz; once a whole card is visible, the app waits 0.6 s for aiming, gives autofocus up to 1.2 s to settle, then requires two sufficiently sharp frames (Laplacian variance on a 256×256 patch of the card's interior, threshold 80) 0.15–0.5 s apart before capturing. None of these preview frames are sent to Photoroom, only the single captured frame is, so automatic capture costs nothing in API calls. Manual capture, library import and the bundled samples remain available; the [README](https://github.com/Photoroom/api-demo-apps/blob/main/iOS%20Perspective%20Correction%20App/README.md#automatic-proportions-and-capture) documents the heuristics in full. #### Step 2: Get a segmentation mask from Photoroom Rather than asking for the cutout, the app asks the Remove Background API for the **segmentation mask** by setting `channels=alpha` (see Download the segmentation mask). The equivalent curl call is: ```bash curl --request POST \ --url https://sdk.photoroom.com/v1/segment \ --header 'x-api-key: YOUR_API_KEY' \ --form image_file=@/path/to/card.jpg \ --form channels=alpha \ --output mask.png ``` In Swift, `Processing/PhotoroomClient.swift` builds the same multipart request by hand with `URLSession`, with no third-party dependency: ```swift var request = URLRequest(url: URL(string: "https://sdk.photoroom.com/v1/segment")!) request.httpMethod = "POST" request.timeoutInterval = 90 request.setValue(apiKey, forHTTPHeaderField: "x-api-key") request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type") request.setValue("image/png", forHTTPHeaderField: "Accept") var body = Data() body.append("--\(boundary)\r\nContent-Disposition: form-data; name=\"image_file\"; filename=\"card.jpg\"\r\nContent-Type: image/jpeg\r\n\r\n") body.append(imageData) body.append("\r\n--\(boundary)\r\nContent-Disposition: form-data; name=\"channels\"\r\n\r\nalpha\r\n") body.append("--\(boundary)--\r\n") request.httpBody = body ``` Why the mask rather than the cutout? The app needs the mask twice: once as a binary image to *find the card's corners*, and once at full resolution to *make the background transparent*. Requesting the grayscale mask gives it both from a single, smaller response. The mask is then used in two ways in `CardProcessor.finish`: * It is downscaled to at most 1,600 pixels and thresholded into a binary image for the geometry step below. * It is composited onto the original photo as its alpha channel (`CIBlendWithAlphaMask`), which removes the background, including around the card's rounded corners, *before* the perspective warp. #### Step 3: Find the corners and straighten the card on-device `Processing/CardGeometry.swift` turns the binary mask into four corners without any machine learning, using only classic image geometry: 1. Flood-fill the background from the image border, so that enclosed holes inside the card don't affect its outline. 2. Keep the largest connected foreground component (the card, or a graded slab if that is what was segmented). 3. Compute its convex hull and simplify it to four significant vertices. 4. Assign the boundary pixels to the four sides and fit a line to each with two passes of robust least squares, which trims rounded corners and stray pixels. 5. Intersect the four lines to get the corners, order them, and reject detections whose confidence or fill ratio is too low. With the corners known, Core Image's `CIPerspectiveCorrection` filter warps the transparent cutout into a rectangle. **Recovering the card's true proportions from the camera** A perspective warp needs to know the target rectangle's width-to-height ratio, and the photo alone cannot tell you: a square seen at an angle and a tall rectangle seen straight on can produce the same four corners. This is where the camera intrinsics from Step 1 come in. `Processing/CameraCalibration.swift` first maps the four corners found on the (rotated, resized) mask back to the unrotated sensor frame, then applies the inverse intrinsic matrix to turn each corner into a viewing ray. For a flat rectangle, the first two columns of the homography from the unit square to those rays are the two physical side vectors, up to a common scale, so the ratio of their lengths is the card's real width/height ratio. The estimate is discarded when the calibration is invalid, when the recovered sides are not close to perpendicular (orthogonality above 0.12), when the ratio falls outside 0.2–5, or when perturbing the corners by a few pixels changes the ratio by more than 6%, which indicates the corners are too imprecise to trust. ```swift guard let base = solve(corners), (0.2...5).contains(base.ratio), base.orthogonality < 0.12 else { return nil } // ... re-solve with slightly perturbed corners ... guard uncertainty < 0.06 else { return nil } return AspectRatioEstimate(widthOverHeight: base.ratio, relativeUncertainty: uncertainty) ``` This is why the **Proportions** control behaves differently depending on where the photo came from. Photos taken in the app have intrinsics, so proportions default to **Automatic**. Photos imported from the library have none, so the app shows apparent proportions as a preview and waits for the user to pick the **Standard card** preset (2.5 : 3.5) or enter **Custom** dimensions before exporting. In both cases the estimate concerns proportions only, not physical size in millimeters. Changing proportions or rotating the result re-renders locally from the retained cutout and mask and **never triggers another API call**. Finally, an on-device text-recognition pass (Vision) compares the four possible orientations and rotates the card upright when printed text clearly favors one of them. #### Step 4 (optional): Blur and relight the background with two Image Editing API calls If the user selects the **Background blur** finish, the app warps the *whole photo* (not just the cutout) so that the card becomes a rectangle while its surroundings are kept, then sends this image to the Image Editing API twice. The first call uses AI Expand to fill the transparent borders that the warp leaves around the photo: ```bash curl --request POST \ --url https://image-api.photoroom.com/v2/edit \ --header 'x-api-key: YOUR_API_KEY' \ --form imageFile=@perspective-corrected-card.png \ --form removeBackground=false \ --form referenceBox=originalImage \ --form expand.mode=ai.auto \ --output expanded.png ``` The second call takes the expanded image and applies Background Blur and AI Relight, exporting a JPEG: ```bash curl --request POST \ --url https://image-api.photoroom.com/v2/edit \ --header 'x-api-key: YOUR_API_KEY' \ --form imageFile=@expanded.png \ --form removeBackground=false \ --form referenceBox=originalImage \ --form background.blur.mode=gaussian \ --form background.blur.radius=0.012 \ --form lighting.mode=ai.preserve-hue-and-saturation \ --form export.format=jpeg \ --output result.jpg ``` A few details worth noting in these two calls: * `removeBackground=false` keeps the original background so it can be expanded, then blurred. Photoroom still detects the subject internally, which is how the blur is applied to the background only. * `referenceBox=originalImage` tells the API to treat the whole canvas, not the cropped subject, as the frame of reference, so the framing of the input image is preserved exactly. * `lighting.mode=ai.preserve-hue-and-saturation` improves the lighting while keeping the card's colours faithful, which matters for collectibles. Turning **Relight** off in the app simply omits this field. In `PhotoroomClient.swift`, the two calls are chained, with a cancellation check between them so that switching finish mid-way doesn't waste a request: ```swift func blurredFinish(image: Data, apiKey: String, relight: Bool = true) async throws -> Data { let expanded = try await send(Self.finishRequest(image: image, apiKey: apiKey, step: .expand)) try Task.checkCancellation() return try await send(Self.finishRequest(image: expanded, apiKey: apiKey, step: .blur, relight: relight)) } ``` After the JPEG comes back, the app crops it to the card's rectified bounds with 5% padding, so the final image is framed the same way as the "Original" it is compared against. ### Handling errors The client maps HTTP status codes to user-facing messages, which is a good pattern to reuse in any mobile integration: ```swift static func responseError(status: Int) -> PhotoroomError { switch status { case 401, 403: .invalidKey // missing or rejected API key case 402: .credits // no API credits left case 429: .rateLimited // default limit is 60 images per minute default: .server(status) } } ``` Each case produces a recoverable error in the UI (a *Retry* button, or a prompt to check the key in Settings), and an in-flight request is cancelled if the user starts a new capture. Timeouts are set generously (90 s for segmentation, 180 s for the AI edits), because AI features on large images can take several seconds. ### Adapting the app to your use case Although the demo is built around trading cards, nothing in the API calls is specific to them. The same pipeline works for any flat rectangular item: comics, vinyl sleeves, book covers, boxed products, ID cards or documents. Here's how to adapt it: * **Different proportions**: the aspect-ratio acceptance range in `CardGeometry.swift` is 1:1 to 5:1, and the *Standard card* preset (2.5 : 3.5) is just a default. Add presets for your item's dimensions. * **A different finish**: the second Image Editing API call is where the look is decided. Replace the blur with a solid `background.color`, an AI Background generated from a `background.prompt`, or add an AI Shadow with `shadow.mode`. Use the *Try it now!* button on each feature page to find the parameters, then paste them into `finishRequest`. * **Server-side processing**: if you'd rather not ship API keys in a mobile app, move `PhotoroomClient` behind your own backend and keep the on-device geometry as is. The API calls are plain multipart HTTP requests and translate directly to any language; see Code Samples. * **Batch processing**: the `Scripts/GenerateReadmeExamples.swift` script in the repository runs the exact same processing code on macOS from the command line, which is a convenient starting point for processing a folder of photos. {% hint style="info" %} Cost per photo: the transparent finish uses **1** Remove Background API call; the blurred finish adds **2** Image Editing API calls. Local adjustments (proportions, rotation, orientation) are free. {% endhint %} ### Going further * Browse the source: [Photoroom/api-demo-apps](https://github.com/Photoroom/api-demo-apps/tree/main/iOS%20Perspective%20Correction%20App) * The repository also contains a [Web demo app](https://github.com/Photoroom/api-demo-apps/tree/main/Web%20Demo%20App) and a general [iOS demo app](https://github.com/Photoroom/api-demo-apps/tree/main/iOS%20Demo%20App). * Related tutorials: How to improve images for second-hand item marketplaces, How to create sticker images. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.photoroom.com/integrations/how-to-correct-the-perspective-of-trading-cards-in-an-ios-app.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.