refactor!: add static factories for remaining models by chmjkb · Pull Request #937 · software-mansion/react-native-executorch

chmjkb · 2026-03-05T12:49:23Z

Description

This PR refactors the existing codebase to match what has been recently done with object detection & semantic segmentation. Namely, it changes the model initialization flow from new Module() -> module.load() to a single, static factory, for example: fromModelName(modelName)

All affected modules now follow the same two-method factory API:

Module.fromModelName(namedSources, onDownloadProgress?) — for built-in models. Requires a modelName field (typed literal union, e.g. LLMModelName, SpeechToTextModelName) that will be used for telemetry once that infrastructure is in place.
Module.fromCustomModel(...sources, onDownloadProgress?) — for user-provided model binaries. Does not expose a modelName parameter — users with custom models shouldn't need to know about or satisfy a type constraint that exists purely for internal tracking.

Note: for now fromCustomModel internally delegates to fromModelName with a 'custom' placeholder. The two methods will diverge meaningfully once telemetry is wired up.

Modules migrated

Module	Notes
`ClassificationModule`	factory + `fromCustomModel`
`StyleTransferModule`	factory + `fromCustomModel`
`ImageEmbeddingsModule`	factory + `fromCustomModel`
`VADModule`	factory + `fromCustomModel`
`TextEmbeddingsModule`	factory + `fromCustomModel`
`OCRModule`	factory + `fromCustomModel`, typed `OCRModelName`
`VerticalOCRModule`	factory + `fromCustomModel`
`LLMModule`	factory + `fromCustomModel`, typed `LLMModelName`
`SpeechToTextModule`	factory + `fromCustomModel`, typed `SpeechToTextModelName`
`TextToImageModule`	factory + `fromCustomModel`, typed `TextToImageModelName`
`ObjectDetectionModule`	`fromCustomConfig` → `fromCustomModel` rename
`SemanticSegmentationModule`	`fromCustomConfig` → `fromCustomModel` rename

TextToSpeechModule is not included — handled separately.

Hook updates

Hooks for modules that have a controller (OCRModule, VerticalOCRModule, LLMModule) already used the controller directly and remain unchanged in structure. Hooks for the remaining migrated modules (useSpeechToText, useTextToImage) were updated to use the factory instead of new Module() + load().

All hooks now include modelName in their useEffect dependency arrays, enabling correct reload behavior when switching between built-in models.

Typed model name unions

Each module now has a corresponding typed union (e.g. LLMModelName, SpeechToTextModelName, OCRModelName) derived from the existing model constants. This replaces the previous use of untyped string and provides compile-time safety when passing model identifiers.

Introduces a breaking change?

Yes
No

Type of change

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Documentation update (improves or adds clarity to existing documentation)
Other (chores, tests, code style improvements etc.)

Tested on

iOS
Android

Testing instructions

Run demo apps and check if they work.
Try to pass model name that is incorrect to check if error is correctly risen.
Try using fromCustomModel method.

Screenshots

Related issues

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

Additional notes

NorbertKlockiewicz

LGTM!

benITo47 · 2026-03-06T18:03:11Z

Code looks good - didn't run it though.

msluszniak · 2026-03-09T12:29:42Z

@chmjkb is there anything to test other then running all the demo apps?

chmjkb · 2026-03-09T12:31:40Z

@chmjkb is there anything to test other then running all the demo apps?

Other than testing the demo apps I think we're good. Maybe checking if type errors occur when unsupported model names are passed would make sense.

msluszniak · 2026-03-10T18:51:04Z

Could we start with a rebase, it will be easier to review it ;p

chmjkb · 2026-03-11T10:57:47Z

Could we start with a rebase, it will be easier to review it ;p

I'll rebase once all the other PRs are merged, so let's wait with the reviews until that happens 😅

benITo47 · 2026-03-11T11:47:15Z

Overall looks good, checked the apps - they work. fromCustomModel also produces expected results.

Fix the docs I marked above, add more info about fromCustomModel to the docs and it's ready to ship!

- Remove abstract load() from BaseModule (enables clean factory subclasses) - Remove no-op load() overrides from BaseLabeledModule and VisionLabeledModule - Add deps parameter to useModuleFactory, relax Config generic constraint - Update useObjectDetection and useSemanticSegmentation to pass explicit deps - Convert ClassificationModule: private constructor + static fromModelName() - Migrate useClassification from useModule to useModuleFactory Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…nd telemetry Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…to fromCustomModel Also fix the misaligned JSDoc on ObjectDetectionModule where the fromCustomConfig doc block was placed before forward() instead of before fromCustomConfig itself. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ddings, VAD, TextEmbeddings Each fromCustomModel delegates to fromModelName with modelName cast to the built-in name type, allowing custom model binaries without requiring a built-in preset name. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…eOCR hook - Add OCRModelName = `ocr-${OCRLanguage}` type to types/ocr.ts - Add modelName: OCRModelName to OCRProps.model - Replace public constructor + load() with private constructor + fromModelName() + fromCustomModel() - Rewrite useOCR to use useModuleFactory with OCRModule.fromModelName Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… controllers - VerticalOCRModule: private constructor, fromModelName, fromCustomModel - useOCR: custom hook using OCRController directly (not useModuleFactory) - useVerticalOCR: custom hook using VerticalOCRController directly Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Add LLMModelName union type - Add modelName to LLMProps.model - LLMModule: private constructor, fromModelName, fromCustomModel - useLLM: add model.modelName to deps Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…tModelName type - Add SpeechToTextModelName union type - Add modelName to SpeechToTextModelConfig - SpeechToTextModule: private constructor, fromModelName, fromCustomModel - useSpeechToText: use factory, add model.modelName to deps Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…odelName type - Add TextToImageModelName union type - Add modelName to TextToImageProps.model - TextToImageModule: private constructor, fromModelName, fromCustomModel - useTextToImage: use factory, add model.modelName to deps Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

NorbertKlockiewicz · 2026-03-12T16:09:42Z

packages/react-native-executorch/src/constants/modelUrls.ts

@@ -763,6 +797,7 @@ export const FCN_RESNET101_QUANTIZED = {
 } as const;

 const SELFIE_SEGMENTATION_MODEL = `${URL_PREFIX}-selfie-segmentation/${NEXT_VERSION_TAG}/xnnpack/selfie-segmentation.pte`;


There's no xnnpack directory so this link is wrong

Suggested change

const SELFIE_SEGMENTATION_MODEL = `${URL_PREFIX}-selfie-segmentation/${NEXT_VERSION_TAG}/xnnpack/selfie-segmentation.pte`;

const SELFIE_SEGMENTATION_MODEL = `${URL_PREFIX}-selfie-segmentation/${NEXT_VERSION_TAG}/selfie-segmentation.pte`;

I think it shoudl be added. So instead of modifying the URL I'll modify the repo so it's aligned iwht other models

packages/react-native-executorch/src/hooks/useModuleFactory.ts

docs/docs/04-typescript-api/01-natural-language-processing/LLMModule.md

…Module.md Co-authored-by: Norbert Klockiewicz <Nklockiewicz12@gmail.com>

NorbertKlockiewicz

LGTM, I run few example apps and didn't encounter any errors.

msluszniak

I got an error when running text to image example:

 ERROR  [TypeError: this.nativeModule.generate is not a function (it is undefined)] 

Code: TextToImageModule.ts
  164 |     seed?: number
  165 |   ): Promise<string> {
> 166 |     const output = await this.nativeModule.generate(
      |                                                    ^
  167 |       input,
  168 |       imageSize,
  169 |       numSteps,
Call Stack
  forward (packages/react-native-executorch/src/modules/computer_vision/TextToImageModule.ts:166:52)
  forward (packages/react-native-executorch/src/modules/computer_vision/TextToImageModule.ts:160:16)
  generate (packages/react-native-executorch/src/hooks/computer_vision/useTextToImage.ts:99:42)
  useTextToImage (packages/react-native-executorch/src/hooks/computer_vision/useTextToImage.ts:81:17)
  runForward (apps/computer-vision/app/text_to_image/index.tsx:56:42)
  TextToImageScreen (apps/computer-vision/app/text_to_image/index.tsx:50:19)
  TouchableOpacity.props.onPress (apps/computer-vision/components/BottomBarWithTextInput.tsx:85:21)

chmjkb · 2026-03-13T10:36:11Z

I got an error when running text to image example:

 ERROR  [TypeError: this.nativeModule.generate is not a function (it is undefined)] 

Code: TextToImageModule.ts
  164 |     seed?: number
  165 |   ): Promise<string> {
> 166 |     const output = await this.nativeModule.generate(
      |                                                    ^
  167 |       input,
  168 |       imageSize,
  169 |       numSteps,
Call Stack
  forward (packages/react-native-executorch/src/modules/computer_vision/TextToImageModule.ts:166:52)
  forward (packages/react-native-executorch/src/modules/computer_vision/TextToImageModule.ts:160:16)
  generate (packages/react-native-executorch/src/hooks/computer_vision/useTextToImage.ts:99:42)
  useTextToImage (packages/react-native-executorch/src/hooks/computer_vision/useTextToImage.ts:81:17)
  runForward (apps/computer-vision/app/text_to_image/index.tsx:56:42)
  TextToImageScreen (apps/computer-vision/app/text_to_image/index.tsx:50:19)
  TouchableOpacity.props.onPress (apps/computer-vision/components/BottomBarWithTextInput.tsx:85:21)

Fixed in 27403e4

chmjkb requested review from NorbertKlockiewicz and benITo47 March 5, 2026 14:06

chmjkb marked this pull request as ready for review March 5, 2026 14:06

chmjkb linked an issue Mar 5, 2026 that may be closed by this pull request

Migrate existing model implementations to use factories instead of legacy flow #938

Closed

chmjkb marked this pull request as draft March 5, 2026 14:19

chmjkb marked this pull request as ready for review March 6, 2026 08:12

NorbertKlockiewicz approved these changes Mar 6, 2026

View reviewed changes

chmjkb mentioned this pull request Mar 6, 2026

feat!: text-to-speech x LLM integration #936

Open

12 tasks

msluszniak assigned chmjkb Mar 9, 2026

msluszniak added the refactoring label Mar 9, 2026

msluszniak marked this pull request as draft March 9, 2026 13:02

chmjkb marked this pull request as ready for review March 10, 2026 10:26

chmjkb requested review from NorbertKlockiewicz and msluszniak March 10, 2026 10:26

chmjkb mentioned this pull request Mar 12, 2026

fix: fix unload in LLM and ModelHostObject to properly free LLM memory #954

Merged

20 tasks

chmjkb and others added 9 commits March 12, 2026 11:16

refactor!: make ImageEmbeddingsModule a factory

b56da84

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

refactor!: make StyleTransferModule a factory

9a042c8

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

refactor!: make VADModule a factory

ac676dd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add modelName to all model constants for union discrimination a…

253eb99

…nd telemetry Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: update JSDoc to document modelName field

22693ca

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

type fixes

ad11843

chmjkb and others added 5 commits March 12, 2026 11:40

chmjkb force-pushed the @chmjkb/static-factories-refactor branch from ba35183 to 86b5021 Compare March 12, 2026 10:56

chmjkb marked this pull request as draft March 12, 2026 10:57

chmjkb added 5 commits March 12, 2026 12:13

lint & fix model names

8845272

docs: update docs, jsdocs

864085f

refactor: refactor TTS

d5f00a7

final chores & docs

a22dd0f

bring back parallel lefthook

a6b1216

chmjkb marked this pull request as ready for review March 12, 2026 15:40

NorbertKlockiewicz reviewed Mar 12, 2026

View reviewed changes

NorbertKlockiewicz requested changes Mar 12, 2026

View reviewed changes

packages/react-native-executorch/src/hooks/useModuleFactory.ts Show resolved Hide resolved

docs/docs/04-typescript-api/01-natural-language-processing/LLMModule.md Show resolved Hide resolved

chmjkb and others added 2 commits March 13, 2026 08:46

fix useModuleFactory

f480e4e

Update docs/docs/04-typescript-api/01-natural-language-processing/LLM…

2d363fc

…Module.md Co-authored-by: Norbert Klockiewicz <Nklockiewicz12@gmail.com>

NorbertKlockiewicz approved these changes Mar 13, 2026

View reviewed changes

msluszniak requested changes Mar 13, 2026

View reviewed changes

fix: await loading the native module

27403e4

msluszniak self-requested a review March 13, 2026 11:17

docs: fix docs

5e87cba

msluszniak approved these changes Mar 13, 2026

View reviewed changes

chmjkb merged commit e827156 into main Mar 13, 2026
5 checks passed

chmjkb deleted the @chmjkb/static-factories-refactor branch March 13, 2026 12:48

		@@ -763,6 +797,7 @@ export const FCN_RESNET101_QUANTIZED = {
		} as const;

		const SELFIE_SEGMENTATION_MODEL = `${URL_PREFIX}-selfie-segmentation/${NEXT_VERSION_TAG}/xnnpack/selfie-segmentation.pte`;

Conversation

chmjkb commented Mar 5, 2026 • edited by msluszniak Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Modules migrated

Hook updates

Typed model name unions

Introduces a breaking change?

Type of change

Tested on

Testing instructions

Screenshots

Related issues

Checklist

Additional notes

Uh oh!

NorbertKlockiewicz left a comment

Choose a reason for hiding this comment

Uh oh!

benITo47 commented Mar 6, 2026

Uh oh!

msluszniak commented Mar 9, 2026

Uh oh!

chmjkb commented Mar 9, 2026

Uh oh!

msluszniak commented Mar 10, 2026

Uh oh!

chmjkb commented Mar 11, 2026

Uh oh!

benITo47 commented Mar 11, 2026

Uh oh!

NorbertKlockiewicz Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

chmjkb Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

NorbertKlockiewicz left a comment

Choose a reason for hiding this comment

Uh oh!

msluszniak left a comment

Choose a reason for hiding this comment

Uh oh!

chmjkb commented Mar 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

chmjkb commented Mar 5, 2026 •

edited by msluszniak

Loading