Skip to content

Conversation

@axelboc
Copy link
Contributor

@axelboc axelboc commented Dec 15, 2025

  • Create Poetry project at support/hsds with hsds and h5pyd
  • Configure HSDS (cf. support/hsds/config)
  • Add --hsds flag to create_h5_sample.py to create sample-hsds.h5 without datasets not supported by hsload
  • Add support:hsds and support:hsds:load scripts to respectively start the HSDS support server and load sample-hsds.h5 into HSDS
  • Create hsds-api.test.ts file and run test to create Vitest snapshot file
  • Update documentation and CI workflow

Opening as draft as I'm having issues in the CI: the HSDS support server won't fully start for some reason. Also, I'm installing and testing the latest version of hsds (and h5pyd), but the demo server on Bosquet still runs an older version.

/cypress/snapshots/**/__diff_output__
/cypress/snapshots/**/__received_output__

/support/hsds/root/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

root is the name of the default bucket,a s configured in support/hsds/config/override.yml

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use a more explicit name e.g. test-bucket

"support:sample": "poetry -C support/sample run python create_h5_sample.py",
"support:h5grove": "poetry -C support/h5grove run python tornado_app.py --basedir ../sample/dist",
"support:hsds": "rm -rf support/hsds/hs.log && poetry -C support/hsds run hsds --root_dir . --config_dir ./config --password_file ./config/passwd.txt",
"support:hsds:load": "rm -rf support/hsds/root/* && poetry -C support/hsds run hsload -e http://localhost:5101 -u test -p test --link ../sample/dist/sample-hsds.h5 /",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The test admin user's password is defined in config/passwd.txt
  • The file is loaded into the root folder of the bucket / and hsload automatically creates a new "domain" for the file called /sample-hsds.h5
  • From what I understand, --link means that the data stays in the file

private readonly baseURL: string,
filepath: string,
private readonly fetcher: Fetcher,
private readonly fetcher = createBasicFetcher(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've managed to configure the HSDS support server to allow unauthenticated requests, so I'm making the fetcher parameter optional so I don't have to pass it in hsds-api.ts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the snapshot file. I've already identified a few issues/inconsistencies with other providers, like a few "Unknown" dtypes, which I'll try to fix.

@axelboc axelboc requested a review from t20100 December 16, 2025 12:39
Copy link
Member

@t20100 t20100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

/cypress/snapshots/**/__diff_output__
/cypress/snapshots/**/__received_output__

/support/hsds/root/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use a more explicit name e.g. test-bucket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants