-
Notifications
You must be signed in to change notification settings - Fork 503
feat: Add LIT integration for interactive model analysis (#121) #1163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
feat: Add LIT integration for interactive model analysis (#121) #1163
Conversation
Release v2.16.2
New Release – v2.17.0
- Fix HookedTransformer type imports to use class from module path - Remove unused imports (TYPE_CHECKING, Optional, Sequence, etc.) - Fix return type annotations for functions returning Optional values - Add proper type: ignore comments for dynamic base classes - Add noqa comments for intentionally unused TYPE_CHECKING imports - Add explicit type annotation for output dict - Format code with ruff formatter
lit-nlp requires Python >=3.9, so add python constraint to ensure compatibility with the project's Python >=3.8 support
…e errors Beartype tries to resolve forward reference strings at runtime, which fails for TYPE_CHECKING-only imports. Replace string type hints like 'HookedTransformer' and 'ActivationCache' with Any to allow runtime type checking to pass while maintaining documentation in docstrings. Also clean up unused imports from test_lit.py: - Remove unused TYPE_CHECKING, Any, Dict, List imports - Remove unused sys and lit_nlp imports - Remove unused inputs variable in test
- Apply black formatting to test_lit.py and __init__.py - Add NBVAL_IGNORE_OUTPUT comment to BERT.ipynb import cell to ignore warnings during import
…n LIT module - Add NBVAL_IGNORE_OUTPUT to Othello_GPT.ipynb import cell (Cell 5) to ignore stderr - Add doctest: +SKIP to all doctest examples in LIT module since lit-nlp is optional - This prevents docstring tests from failing when LIT is not installed
…opes_Generation_Demo notebooks
…Analysis_Demo.ipynb
|
Note on the large diff (+26k / -20k changes): This was necessary to resolve GitHub Action failures where nbval was failing due to environmental warnings (stderr) in the CI runner. Because .ipynb files are large JSON documents, adding even a single comment line causes significant changes in the underlying file structure and metadata. The actual code logic for the LIT integration is localized and can be reviewed primarily in the ' transformer_lens/lit/ 'directory. |
Description
Implements Google's Learning Interpretability Tool (LIT) integration for TransformerLens, allowing interactive model analysis and visualization.
Features:
HookedTransformerLITwrapper implementing the full LIT Model APIserve()function to launch LIT serverUsage:
LIT is an optional dependency - existing installations are unaffected.
Fixes #121
Type of change
Checklist: