indexers.VectorStore.from_filespace
indexers.VectorStore.from_filespace(
folder_path,
vectoriser,
batch_size=None,
hooks=None,
quiet_mode=False,
)Creates a VectorStore instance from a saved filespace folder.
Reads metadata.json and vectors.parquet from folder_path using fsspec, so both local and remote paths (e.g. gs://) are supported. The vectoriser class name stored in metadata.json must match the class name of the supplied vectoriser object. The instance is constructed via object.__new__, so init is never called and no embeddings are generated.
Note: the returned instance does not have output_dir or skip_save attributes set. vector_shape and num_vectors are read directly from metadata.json without being cross-checked against the actual contents of the parquet file.
Loading a VectorStore whose metadata.json was produced by v1.0.0 (which did not persist batch_size) will raise a DataValidationError because batch_size is listed as a required metadata key. Passing batch_size as an argument does not bypass this check.
Fix: upgrade to v1.1.1+, which resolves this issue.
Workaround: If you are unable to update from v1.1.0 to a later version, this issue may be circumvented by manually editing the metadata.json file to include a batch_size field (e.g., "batch_size": 128). We advise against this approach. Editing metadata.json directly is not recommended as a general practice and can lead to issues.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| folder_path | str | Path to the folder containing metadata.json and vectors.parquet. Supports any fsspec-compatible path (local, gs://, etc.). | required |
| batch_size | int | None | Overrides the batch_size stored in metadata. Defaults to None, which uses the value from metadata.json. | None |
| vectoriser | An object with a callable .transform(texts) method. Its class name must match the vectoriser_class value stored in metadata.json. | required | |
| hooks | dict | None | A dictionary of user-defined hooks for preprocessing and postprocessing. Defaults to None. | None |
| quiet_mode | bool | Whether to minimise verbose output, such as progress bars. Defaults to False. | False |
Returns
| Name | Type | Description |
|---|---|---|
| VectorStore | A VectorStore instance with vectors populated from the parquet file. file_name, data_type, and batch_size are all set to None. |
Raises
| Name | Type | Description |
|---|---|---|
| DataValidationError | If folder_path is not a non-empty string, does not point to an existing directory, if metadata.json is missing or malformed, or if vectors.parquet is missing, empty, or does not contain the required columns. | |
| OptionalDependencyError | If the user attempts to use a gs:// path without having gcsfs installed. | |
| ConfigurationError | If vectoriser does not have a callable .transform() method, if the fsspec path cannot be resolved, or if the vectoriser class name does not match the one stored in metadata.json. | |
| IndexBuildError | If metadata.json or vectors.parquet cannot be read or parsed, or if the instance cannot be constructed. |