Shapes
On-device single-stroke shape recognition.
Shapes model page- Platforms
- iOS, macOS, tvOS, visionOS, Android, Linux, Windows, Browser, Node
- Weights
- v0.3.0
Install
Swift
.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "3.5.0")
Then add the Shapes product to your target.
Kotlin
implementation("ai.desertant:shapes:3.5.0")
Terminal
npm i @desert-ant-labs/shapes @litertjs/core # browser
npm i @desert-ant-labs/shapes # Node, prebuilt native core
Usage
Swift
import Shapes
let shapes = Shapes()
if let shape = try await shapes.recognize(points: strokePoints) {
switch shape {
case let .rectangle(corners): ... // [Point]
case let .ellipse(center, semiMajor, semiMinor, rotation): ...
default: break
}
}
recognize accepts [Point] or, on Apple platforms, [CGPoint] and PencilKit
PKStroke; Shape.path gives a renderable CGPath. On iOS and visionOS, live
snapping on a PencilKit canvas is one line. Pausing mid-stroke previews the
recognized shape, lifting the pen swaps it in, and the swap is registered with
the canvas's undo manager:
Swift
canvasView.enableShapeSnapping()
Kotlin
import ai.desertant.shapes.Point
import ai.desertant.shapes.Shape
import ai.desertant.shapes.Shapes
Shapes(context).use { shapes ->
when (val shape = shapes.recognize(strokePoints)) { // Shape? (null if rejected)
is Shape.Rectangle -> shape.corners
is Shape.Ellipse -> shape.center
else -> {}
}
}
TypeScript
import { Shapes } from "@desert-ant-labs/shapes"; // browser
// import { Shapes } from "@desert-ant-labs/shapes/native"; // server-side Node
const shapes = await Shapes.load();
const shape = await shapes.recognize(points); // [{x, y}, ...] or [x0, y0, ...]
if (shape?.kind === "ellipse") shape.center; // null when the stroke is rejected
shapes.dispose();
Loading the model
The weights are fetched from the Hub on first use and cached. See model downloads and caching.
Files
| File | Format | Size | Contents |
|---|---|---|---|
shapes. | LiteRT / TFLite (fp32) | ~1.3 MB | Fixed [1, features + [1, mask window; runs on Android, Linux, Node, and the web (bundled by default in the Kotlin SDK; downloaded on demand by the JavaScript SDK) |
shapes. | Compiled Core ML | ~0.2 MB | 4-bit-palettized classifier, ready to load on Apple platforms (used by the Swift SDK) |
shapes_ | JSON | tiny | classes, preprocessing constants, model dims, and snap gates |
shapes. | safetensors | ~0.2 MB | Packed weights for SDK versions before the LiteRT migration (tag v0.) |
Inputs and outputs
- Input: an ordered list of stroke points in canvas coordinates. Single stroke.
- Output: a shape class plus fitted geometry, or nothing if the stroke is rejected.
Classes
line, rectangle, triangle, ellipse, star, plus none (the reject class:
scribbles, partial shapes, and other non-shape strokes). Squares and circles are
covered by rectangle and ellipse (snapped when near-regular).
Limitations
- Single stroke only; multi-stroke shapes aren't recognized.
- Tuned for deliberate shapes; very rough or ambiguous strokes are rejected by design.