vh/heretic
1
0
Fork 0
mirror of https://github.com/p-e-w/heretic.git synced 2026-09-25 13:31:33 -07:00
Code Issues Packages Projects Releases Wiki Activity
196 Commits 13 Branches 5 Tags
version-2-dev
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Philipp Emanuel Weidmann 3444a0dd67 Merge branch 'master' into version-2-dev 2026-08-17 16:20:28 +05:30
Philipp Emanuel Weidmann 346d61673b chore: remove Gemini style guide (#427) 2026-08-17 16:19:58 +05:30
Philipp Emanuel Weidmann b796257c37 chore: bump version to 2.0.0.dev0 2026-08-17 15:23:36 +05:30
Philipp Emanuel WeidmannandVinay Umrethe 0146b2760f feat: headless operation + end-to-end tests (#392) 
* fix: remove notebook input shims

Closes #280

* feat: support headless operation (no interactive input)

* fix: prevent infinite loops

* feat: add end-to-end tests

* ci: run tests in CI

* ci: fix test output ordering

* fix: replace home-cooked `set_seed` function with Transformers builtin

* feat: print PyTorch config when running tests

* feat: print additional information

* experiment: try to standardize test environment

* fix: revert environment changes

* feat: support multiple valid hashes for each output file

* feat: add test output hashes for CI

* feat: add test output hashes for CI (alternative environment)

* feat: add hashes for Windows (#394)

* fix: Hash on windows

* trigger ci

* fix: prefer .yaml (used widely than .toml for model configs)

* use removeprefix

* docs: restore commet

* use removeprefix again

* tests: Add windows hash files for all test models

* trigger ci

* fix: minor cleanup

* clean merge mismatch

* remove unnecessary CRLF replace, now that we support more SUMS files

* fix: use binary mode for hashes everywhere

---------

Co-authored-by: Vinay Umrethe <umrethevinay@gmail.com>
 2026-06-27 13:41:48 +05:30
Philipp Emanuel Weidmann 6ea3b8d778 build: bump version to 1.4.0 2026-06-14 16:37:45 +05:30
Philipp Emanuel Weidmann 6757ada999 fix: minor cleanups and improvements 2026-06-13 19:48:38 +05:30
Philipp Emanuel WeidmannandVinay-Umrethe 2fd163f5e4 feat: automatically reproduce model from reproduce.json (#326) 
* feat: load reproduction information

* feat: check reproduction environment against original environment

* fix: remove `trust_remote_code` setting

This improves security when running Heretic with an untrusted config file. The prompt is now always shown.

This is NOT a breaking change, because we currently ignore values for unknown settings, so existing configs continue to work.

* feat: reproduce model from JSON file

* feat: verify hashes of uploaded weight files

* fix: fix issues in automatic reproduction system (#352)

* fix: Check if a model is gated / accessible

* fix: handle unknown gated models

* feat: Auto install requirements

* simplify

* Revert "simplify"

This reverts commit 10287926e9.

* Revert "feat: Auto install requirements"

This reverts commit f4be1abd04.

* fix: Seed pytorch method

* reference, style

* simplify token

* feat: Export strategy in reproduce.json, v2

* style: Name

* simplify export strategy

* style: Rename

* enumeration

* maybe remove seed as well

* fix: don't lock settings with permanent strategy

* simplify no choice, use try/finally block

* feat: verify hashes of locally saved weight files

* fix: remove obsolete code from merge

* docs: add automatic reproduction instructions to reproduce README

---------

Co-authored-by: Vinay-Umrethe <vinayumrethe99@gmail.com>
 2026-06-11 14:49:28 +05:30
Philipp Emanuel Weidmann c62e10d570 fix: install kernels as a Transformers extra 
Fixes #343
 2026-06-04 12:17:35 +05:30
Philipp Emanuel Weidmann 4e3a3a78a3 docs: update README 2026-05-22 14:51:24 +05:30
Philipp Emanuel Weidmann b2bdc1f9d6 feat: add functionality for collecting reproduce.json files from Hugging Face 2026-05-07 18:33:50 +05:30
Philipp Emanuel Weidmann 9b7624ddfa build: bump version to 1.3.0 2026-05-05 18:22:02 +05:30
Philipp Emanuel Weidmann 0e7c14d94a fix: minor cleanups and improvements 2026-05-04 22:11:14 +05:30
Philipp Emanuel Weidmann 02ce8ad079 chore: update dependencies 2026-05-03 19:25:36 +05:30
Philipp Emanuel Weidmann 79ea9ce905 docs: update README 2026-05-03 09:08:57 +05:30
Philipp Emanuel Weidmann 216c089974 Revert "Revert "fix: disable LoRA export for now" (#308)" (#319) 
This reverts commit da92f745de.
 2026-05-03 07:25:00 +05:30
Philipp Emanuel Weidmann 43f8e86a84 fix: minor cleanups and improvements 2026-05-02 06:35:31 +05:30
Philipp Emanuel Weidmann 513e3acc72 fix: improve the reproducibility system (#303) 
* fix: various cleanups and improvements for the reproducibility system

* fix: save only essential settings

* fix: improve model commit handling

* feat: make including system information optional

* fix: improve formatting of reproducibility README

* fix: fix remaining issues
 2026-04-23 19:08:18 +05:30
Philipp Emanuel Weidmann b08a0925c1 feat: make response prefix logic configurable 2026-04-07 13:24:48 +05:30
Philipp Emanuel Weidmann f612a48b9f build: prevent installing dependency packages published in the past 7 days 2026-04-04 08:54:37 +05:30
Philipp Emanuel Weidmann 887d43a8d9 fix: set batch size on HFLM object 2026-04-01 14:27:43 +05:30
Philipp Emanuel Weidmann 96c7a7d98a fix: replace tqdm progress bars with Rich progress bars 2026-03-28 18:30:15 +05:30
Philipp Emanuel Weidmann 1126332281 feat: add integrated benchmarking system 2026-03-24 18:25:12 +05:30
Philipp Emanuel Weidmann 19cdf7e244 fix: address ty complaint 2026-03-15 09:58:00 +05:30
Philipp Emanuel Weidmann 94775d4148 chore: update dependencies 2026-03-15 09:31:32 +05:30
Philipp Emanuel Weidmann ec0367226d style: fix formatting and naming 2026-03-06 13:18:08 +05:30
Philipp Emanuel Weidmann cb4ef3fdfc docs: add Trendshift badge to README 2026-02-20 13:00:19 +05:30
Philipp Emanuel Weidmann 27097bfe8e build: bump version to 1.2.0 2026-02-14 18:11:42 +05:30
Philipp Emanuel Weidmann 025ab3a881 fix: disable LoRA export for now 
Workaround for #152
 2026-02-14 16:56:12 +05:30
Philipp Emanuel Weidmann 1179013999 docs: update README 2026-02-14 16:32:08 +05:30
Philipp Emanuel Weidmann fe7bc1bae3 docs: update README 2026-02-14 10:47:28 +05:30
Philipp Emanuel Weidmann e70a1a85e8 fix: don't load checkpoint when evaluating a second model 
Fixes #144
 2026-02-14 10:02:17 +05:30
Philipp Emanuel Weidmann e7f8be98b7 fix: only export tokenizer when exporting full model 
Fixes #143
 2026-02-14 09:18:22 +05:30
Philipp Emanuel Weidmann 6017bcd347 fix: use compatible release specifiers for non-dev dependencies 
Fixes #145

Credit to MuX on Discord for recognizing that this is an issue with Transformers 5
 2026-02-13 12:27:57 +05:30
Philipp Emanuel Weidmann dd0b3a2f69 docs: update README 2026-02-11 11:09:17 +05:30
Philipp Emanuel Weidmann b873598b77 docs: improve settings documentation 2026-02-11 10:19:05 +05:30
Philipp Emanuel Weidmann 10ceb3098e chore: update copyright notice 2026-02-11 09:46:36 +05:30
Philipp Emanuel Weidmann f68a887a7b fix: improve code quality, improve UX, fix small bugs 2026-02-08 13:32:00 +05:30
Philipp Emanuel Weidmann 2690655a83 feat: print memory usage during run 2026-02-02 21:18:01 +05:30
Philipp Emanuel Weidmann 632b1da622 feat: add config file for slop reduction 2026-01-11 18:51:26 +05:30
Philipp Emanuel Weidmann 1cfd09d7f3 ci: add style guide for Gemini 2026-01-09 14:58:56 +05:30
Philipp Emanuel Weidmann 09be09e12e fix: restore classification of empty responses as refusals 
Fixes #93
 2026-01-02 16:50:02 +05:30
Philipp Emanuel Weidmann 039f6222d2 feat: allow overriding the system prompt per dataset 2025-12-31 14:26:44 +05:30
Philipp Emanuel Weidmann c4b2ea0c42 feat: allow injecting prefixes and suffixes into prompts 2025-12-31 12:00:44 +05:30
Philipp Emanuel Weidmann 02a5237a02 feat: add option to print prompt/response pairs 2025-12-27 14:48:29 +05:30
Philipp Emanuel Weidmann cf8cf6f349 fix: address remaining ty complaint 2025-12-22 11:12:45 +05:30
Philipp Emanuel Weidmann 2141e110fb ci: treat ty warnings as errors 2025-12-22 10:57:36 +05:30
Philipp Emanuel Weidmann 39101137ef ci: add type checking 2025-12-22 10:48:42 +05:30
Philipp Emanuel Weidmann 064bed9a9f fix: resolve issues raised by ty 
A single issue has been deliberately left unfixed to verify that the CI check works
 2025-12-22 10:24:55 +05:30
Philipp Emanuel Weidmann 5ddef6fd2f feat: add more CoT templates 
Suggested by u/Chromix_ on Reddit
 2025-12-20 17:12:46 +05:30
Philipp Emanuel Weidmann d9f2b0407a build: bump version to 1.1.0 2025-12-10 16:54:03 +05:30
Philipp Emanuel Weidmann ca783db6c9 docs: update README 2025-12-10 16:30:35 +05:30
Philipp Emanuel Weidmann 6acccac994 feat: add progress bars for plotting operations 2025-12-10 13:07:34 +05:30
Philipp Emanuel Weidmann ac154a55a0 fix: suppress CoT output for thinking models 
Ref #75
 2025-12-09 11:54:08 +05:30
Philipp Emanuel Weidmann 15781a8a0c fix: skip common response prefix for thinking models 
Ref #75
 2025-12-09 08:25:10 +05:30
Philipp Emanuel Weidmann 24c3aeb442 feat: turn boolean settings into CLI flags 2025-12-07 11:37:07 +05:30
Philipp Emanuel Weidmann ffbde3ac2a fix: follow up after recent PRs 2025-12-07 10:26:16 +05:30
Philipp Emanuel Weidmann 932d737edf feat: add silhouette coefficient to residual geometry output 2025-12-07 08:48:38 +05:30
Philipp Emanuel Weidmann 1f5e977f4f Revert "perf: optimize abliteration matrix op (#46)" (#74) 
This reverts commit 60bd531fde.
 2025-12-07 06:30:37 +05:30
Philipp Emanuel Weidmannandarnomatic da27ba8054 fix: always left-pad inputs, and avoid optimizing for empty responses 
Fixes #70

Co-authored-by: arnomatic <acc@eml.cc>
 2025-12-06 06:31:09 +05:30
Philipp Emanuel Weidmann baf5b0b0d1 feat: add geometric median to residual geometry output 2025-12-05 20:15:50 +05:30
Philipp Emanuel Weidmann eeb28b28c1 feat: add option to plot residual vectors 2025-12-04 14:22:29 +05:30
Philipp Emanuel Weidmann 83cbf0612a Add option to print refusal geometry 2025-11-22 13:18:54 +05:30
Philipp Emanuel Weidmann c35f3031f8 Allow stopping the optimization process early with Ctrl+C 2025-11-21 10:11:00 +05:30
Philipp Emanuel Weidmann 22a4a5b5b5 Add citation information to README 2025-11-19 12:14:17 +05:30
Philipp Emanuel Weidmann 694edf18d3 Follow up after recent PRs 2025-11-19 11:19:47 +05:30
Philipp Emanuel Weidmann c9c022a143 Fix linting issues 2025-11-19 10:16:58 +05:30
Philipp Emanuel Weidmann 9905d9517f Fix formatting issues 2025-11-19 10:04:43 +05:30
Philipp Emanuel Weidmann f06e939791 Add Ruff as a dev dependency 2025-11-19 09:59:18 +05:30
Philipp Emanuel Weidmann f3b9826ca4 Add CI workflow 2025-11-19 09:45:54 +05:30
Philipp Emanuel Weidmann b3545e4b1e Fix retrieving package version 2025-11-16 17:35:13 +05:30
Philipp Emanuel Weidmann 3f346b6150 Change package name 2025-11-16 17:01:50 +05:30
Philipp Emanuel Weidmann 1a59d226c1 Fix spacing after images in README 2025-11-16 16:06:08 +05:30
Philipp Emanuel Weidmann 12ecf50033 Add README 2025-11-16 15:19:27 +05:30
Philipp Emanuel Weidmann ea699dce46 Improve appearance of selection menus 2025-11-16 11:32:58 +05:30
Philipp Emanuel Weidmann 8a1aceff11 Switch to multi-objective optimization 2025-11-14 18:04:23 +05:30
Philipp Emanuel Weidmann 0bae27f359 Fix some of the problems with Falcon-E-3B 2025-11-13 11:39:08 +05:30
Philipp Emanuel Weidmann e24080db64 Add metadata to pyproject.toml 2025-11-02 10:06:15 +05:30
Philipp Emanuel Weidmann fae39ffb89 Move default configuration to Python 2025-11-02 09:29:55 +05:30
Philipp Emanuel Weidmann 850c21b534 Make multivariate TPE work properly 2025-11-01 16:57:12 +05:30
Philipp Emanuel Weidmann a24e6eba96 Improve optimization 2025-10-31 16:04:28 +05:30
Philipp Emanuel Weidmann a9655c8d31 Perform calculations involving residual vectors in float32 
Credit to Jim Lai for pointing out potential numerical problems in https://huggingface.co/blog/grimjim/projected-abliteration
 2025-10-31 13:47:24 +05:30
Philipp Emanuel Weidmann 1496e0a04c Dynamically choose between global and per-layer refusal directions 2025-10-31 13:04:45 +05:30
Philipp Emanuel Weidmann c638d3d012 Adjust score parameters 2025-10-25 13:15:31 +05:30
Philipp Emanuel Weidmann 47e855d5d8 Guard against missing model card data 2025-10-25 13:12:18 +05:30
Philipp Emanuel Weidmann e2419de016 Add "abliterated" to model tags 2025-10-25 09:59:44 +05:30
Philipp Emanuel Weidmann ad8b04d371 Bump version to 1.0.0 2025-10-25 09:52:43 +05:30
Philipp Emanuel Weidmann 37c5ea06d1 Print elapsed and remaining time 2025-10-25 09:47:54 +05:30
Philipp Emanuel Weidmann cf57a0cfbe Add functionality to evaluate any model relative to the main model 2025-10-24 13:38:03 +05:30
Philipp Emanuel Weidmann e6aba71186 Improve refusal detection 2025-10-24 11:27:28 +05:30
Philipp Emanuel Weidmann f8f3f9a012 Fix chat responses being cut off 2025-10-22 12:30:28 +05:30
Philipp Emanuel Weidmann 6359aa44bb Separate abliteration parameters for different layer components 2025-10-22 12:05:28 +05:30
Philipp Emanuel Weidmann ed65d6902b Support gpt-oss MoE 2025-10-15 17:51:39 +05:30
Philipp Emanuel Weidmann 7ed0cb1ffb Support Phi-3.5-MoE 2025-10-14 11:23:53 +05:30
Philipp Emanuel Weidmann 8b827ee386 Support multimodal models 2025-10-14 10:32:34 +05:30
Philipp Emanuel Weidmann dd7abd3296 Add hf_transfer to dependencies 
Required for repositories that don't use Xet
 2025-10-14 07:56:43 +05:30
Philipp Emanuel Weidmann 3d5e645c13 Handle Ctrl+C gracefully 2025-10-12 12:53:40 +05:30
Philipp Emanuel Weidmann 74b55977f0 Pretty-print configuration errors 2025-10-12 10:39:59 +05:30
Philipp Emanuel Weidmann b4a0c0d3f2 Add program version to generated README intro 2025-10-11 17:31:11 +05:30
Philipp Emanuel Weidmann 7caf9fcdc5 Separate training and evaluation prompts 2025-10-09 12:51:31 +05:30
Philipp Emanuel Weidmann 2ff8dcba6b Add model card when uploading to Hugging Face 2025-09-30 08:43:21 +05:30