Skip to content

load Vocabulary from local path #226

@ImpleLee

Description

@ImpleLee

What behavior of the library made you think about the improvement?

I have a local tokenizer at, let's say, /tmp/tokenizer1/. I want to use it with outlines-core.

I tried Vocabulary.from_pretrained('/tmp/tokenizer1/'), but it said it couldn't find the tokenizer at huggingface.

How would you like it to behave?

Support local tokenizer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions