Desert Ant Labs

Shapes

On-device single-stroke shape recognition.

Shapes model page
Platforms
iOS, macOS, tvOS, visionOS, Android, Linux, Windows, Browser, Node
Weights
v0.3.0

Install

requirements

Swift
.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "3.5.0")

Then add the Shapes product to your target.

requirements

Kotlin
implementation("ai.desertant:shapes:3.5.0")

requirements

Terminal
npm i @desert-ant-labs/shapes @litertjs/core   # browser
npm i @desert-ant-labs/shapes                  # Node, prebuilt native core

Usage

Swift
import Shapes

let shapes = Shapes()
if let shape = try await shapes.recognize(points: strokePoints) {
    switch shape {
    case let .rectangle(corners): ...       // [Point]
    case let .ellipse(center, semiMajor, semiMinor, rotation): ...
    default: break
    }
}

recognize accepts [Point] or, on Apple platforms, [CGPoint] and PencilKit PKStroke; Shape.path gives a renderable CGPath. On iOS and visionOS, live snapping on a PencilKit canvas is one line. Pausing mid-stroke previews the recognized shape, lifting the pen swaps it in, and the swap is registered with the canvas's undo manager:

Swift
canvasView.enableShapeSnapping()
Kotlin
import ai.desertant.shapes.Point
import ai.desertant.shapes.Shape
import ai.desertant.shapes.Shapes

Shapes(context).use { shapes ->
    when (val shape = shapes.recognize(strokePoints)) {   // Shape? (null if rejected)
        is Shape.Rectangle -> shape.corners
        is Shape.Ellipse -> shape.center
        else -> {}
    }
}
TypeScript
import { Shapes } from "@desert-ant-labs/shapes";       // browser
// import { Shapes } from "@desert-ant-labs/shapes/native"; // server-side Node

const shapes = await Shapes.load();
const shape = await shapes.recognize(points);   // [{x, y}, ...] or [x0, y0, ...]
if (shape?.kind === "ellipse") shape.center;    // null when the stroke is rejected
shapes.dispose();

Loading the model

The weights are fetched from the Hub on first use and cached. See model downloads and caching.

Files

FileFormatSizeContents
shapes.tfliteLiteRT / TFLite (fp32)~1.3 MBFixed [1,256,3] features + [1,256] mask window; runs on Android, Linux, Node, and the web (bundled by default in the Kotlin SDK; downloaded on demand by the JavaScript SDK)
shapes.mlmodelcCompiled Core ML~0.2 MB4-bit-palettized classifier, ready to load on Apple platforms (used by the Swift SDK)
shapes_meta.jsonJSONtinyclasses, preprocessing constants, model dims, and snap gates
shapes.safetensorssafetensors~0.2 MBPacked weights for SDK versions before the LiteRT migration (tag v0.1.0)

Inputs and outputs

  • Input: an ordered list of stroke points in canvas coordinates. Single stroke.
  • Output: a shape class plus fitted geometry, or nothing if the stroke is rejected.

Classes

line, rectangle, triangle, ellipse, star, plus none (the reject class: scribbles, partial shapes, and other non-shape strokes). Squares and circles are covered by rectangle and ellipse (snapped when near-regular).

Limitations

  • Single stroke only; multi-stroke shapes aren't recognized.
  • Tuned for deliberate shapes; very rough or ambiguous strokes are rejected by design.