Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Option to split during conversion #6942

Open
wants to merge 71 commits into
base: master
Choose a base branch
from
Open

Option to split during conversion #6942

wants to merge 71 commits into from
+223 −74

Commits on Apr 27, 2024

  1. support splits in convert.py

    @christianazinn
    christianazinn committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    874c341 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2024

  1. Support split by size and dry run to write estimated shards/filesizes

    @christianazinn
    christianazinn committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    72cbd4e View commit details
    Browse the repository at this point in the history

  2. Move split functionality to new GGUFManager class

    @christianazinn
    christianazinn committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    702a744 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. fix improper function signature

    @christianazinn
    christianazinn committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    c33bdf3 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2024

  1. tentative push of convert-hf-to-gguf support

    @christianazinn
    christianazinn committed May 5, 2024
    Configuration menu
    Copy the full SHA
    b7c6120 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Merge branch 'master' into convert-split

    @mofosyne
    mofosyne committed May 9, 2024
    Configuration menu
    Copy the full SHA
    14b3291 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. resolve merge + SplitArguments for easier parsing

    @christianazinn
    christianazinn committed May 10, 2024
    Configuration menu
    Copy the full SHA
    87a98a5 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. Merge remote-tracking branch 'origin' into convert-split

    @christianazinn
    christianazinn committed May 23, 2024
    Configuration menu
    Copy the full SHA
    2dd7841 View commit details
    Browse the repository at this point in the history

  2. Fix eager tensor memory leak and remove convert.py changes 

    Removed a memory leak caused by unexpected reference retention to eager tensors.

Also removed GGUFManager functionality in convert.py in favor of specializing for convert-hf-to-gguf.py.
    @christianazinn
    christianazinn committed May 23, 2024
    Configuration menu
    Copy the full SHA
    3ff27ef View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. refactor SplitStrategy to be a deque 

    Instead of having SplitStrategy have a `data` field that is a deque, just have SplitStrategy be a subclass of deque itself.
    @christianazinn
    christianazinn committed May 24, 2024
    Configuration menu
    Copy the full SHA
    6b5c375 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. fix Q8 quantization

    @christianazinn
    christianazinn committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    09baf2f View commit details
    Browse the repository at this point in the history

  2. remove unnecessary imports in gguf_manager

    @christianazinn
    christianazinn committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    240243e View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into convert-split

    @christianazinn
    christianazinn committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    140eb52 View commit details
    Browse the repository at this point in the history

  4. fix final? merge issue

    @christianazinn
    christianazinn committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    a9c7703 View commit details
    Browse the repository at this point in the history

  5. fix gguf_writer placement and remove comments

    @christianazinn
    christianazinn committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    efead04 View commit details
    Browse the repository at this point in the history

  6. oops, actually fix gguf_writer placement

    @christianazinn
    christianazinn committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    c8ecbc6 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. reduce duplicated code from gguf_writer

    @christianazinn
    christianazinn committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    3e9430d View commit details
    Browse the repository at this point in the history

  2. further simplify GGUFManager

    @christianazinn
    christianazinn committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    f6fd3ea View commit details
    Browse the repository at this point in the history

  3. simplify even further and standardize with GGUFWriter

    @christianazinn
    christianazinn committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    bb5ee02 View commit details
    Browse the repository at this point in the history

  4. reduce diffs with master

    @christianazinn
    christianazinn committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    5ad397d View commit details
    Browse the repository at this point in the history

  5. form shards while adding tensors, SHA256 sums agree with master

    @christianazinn
    christianazinn committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    ce7e698 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. re-add type hint 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    706bd69 View commit details
    Browse the repository at this point in the history

  2. GGUFWriter compatibility fix 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    6a05183 View commit details
    Browse the repository at this point in the history

  3. Shard dataclass and un-negative dont_add_architecture

    @christianazinn
    christianazinn committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    3328b0a View commit details
    Browse the repository at this point in the history

  4. type consistency in format_n_bytes_to_str

    @christianazinn
    christianazinn committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    1cbab22 View commit details
    Browse the repository at this point in the history

  5. move kv keys to constants.py

    @christianazinn
    christianazinn committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    2037eab View commit details
    Browse the repository at this point in the history

  6. make pathlib explicit

    @christianazinn
    christianazinn committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    83e4a3f View commit details
    Browse the repository at this point in the history

  7. base-1024 bytes to base-1000

    @christianazinn
    christianazinn committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    13ffe22 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. rename GGUFManager to GGUFWriterSplit

    @christianazinn
    christianazinn committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    6d3a256 View commit details
    Browse the repository at this point in the history

  2. Update gguf-py/gguf/constants.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    1312e28 View commit details
    Browse the repository at this point in the history

  3. fix convert-hf-to-gguf.py permissions

    @christianazinn
    christianazinn committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    5f29d4a View commit details
    Browse the repository at this point in the history

  4. fix line endings

    @christianazinn
    christianazinn committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    0283fc1 View commit details
    Browse the repository at this point in the history

  5. Update gguf-py/gguf/gguf_writer_split.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    dc5cf5f View commit details
    Browse the repository at this point in the history

  6. convert-hf : restore executable file permission

    @compilade
    compilade committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    e093dfb View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2024

  1. examples/convert-legacy-llama.py: restore executable file permission

    @christianazinn
    christianazinn committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    9576965 View commit details
    Browse the repository at this point in the history

  2. reinstate original gguf package import and fix type annotation

    @christianazinn
    christianazinn committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    c6ae1d6 View commit details
    Browse the repository at this point in the history

  3. attempt to appease the linter

    @christianazinn
    christianazinn committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    2e70fa1 View commit details
    Browse the repository at this point in the history

  4. attempt 2 to appease the linter

    @christianazinn
    christianazinn committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    891b19c View commit details
    Browse the repository at this point in the history

  5. attempt 3 to appease the linter

    @christianazinn
    christianazinn committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    02be0dd View commit details
    Browse the repository at this point in the history

  6. comma consistency

    @christianazinn
    christianazinn committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    f658e91 View commit details
    Browse the repository at this point in the history

  7. Update convert-hf-to-gguf.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    079dfe3 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2024

  1. edit cmd line args

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    282e71f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into convert-split

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    666bb09 View commit details
    Browse the repository at this point in the history

  3. use simplification from ggerganov#7827

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    03cc9bc View commit details
    Browse the repository at this point in the history

  4. kv/ti data are still wrong

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    97dd416 View commit details
    Browse the repository at this point in the history

  5. try to refactor kv data (still fails)

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    ff2dd7d View commit details
    Browse the repository at this point in the history

  6. fix ti data messiness

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    ba1be97 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' into convert-split

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    69d6e7a View commit details
    Browse the repository at this point in the history

  8. tidy up

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    0779f2f View commit details
    Browse the repository at this point in the history

  9. fix linting

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    a234bf8 View commit details
    Browse the repository at this point in the history

  10. actually make the linter happy

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    49b9fbe View commit details
    Browse the repository at this point in the history

  11. cleanup round 1

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    0471f67 View commit details
    Browse the repository at this point in the history

  12. remove SplitStrategy, SplitArguments

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    5a96b8f View commit details
    Browse the repository at this point in the history

  13. appease linter

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    f7ecd99 View commit details
    Browse the repository at this point in the history

  14. fix typing and clean up

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    9d7f694 View commit details
    Browse the repository at this point in the history

  15. fix linting

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    0417104 View commit details
    Browse the repository at this point in the history

  16. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    70a6bc9 View commit details
    Browse the repository at this point in the history

  17. progress bar, fix split logic

    @christianazinn
    christianazinn committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    1e2d9cb View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    f7e7983 View commit details
    Browse the repository at this point in the history

  2. catch oversights

    @christianazinn
    christianazinn committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    79bd2bf View commit details
    Browse the repository at this point in the history

  3. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    7eea552 View commit details
    Browse the repository at this point in the history

  4. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    99f9a24 View commit details
    Browse the repository at this point in the history

  5. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    ad02c94 View commit details
    Browse the repository at this point in the history

  6. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    c1b1a29 View commit details
    Browse the repository at this point in the history

  7. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    4550826 View commit details
    Browse the repository at this point in the history

  8. swap bar orders

    @christianazinn
    christianazinn committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    efa0609 View commit details
    Browse the repository at this point in the history

  9. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    b843445 View commit details
    Browse the repository at this point in the history

  10. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    854bd64 View commit details
    Browse the repository at this point in the history

  11. compatibility fix

    @christianazinn
    christianazinn committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    05b183f View commit details
    Browse the repository at this point in the history

  12. Update gguf-py/gguf/gguf_writer.py 

    Co-authored-by: compilade <[email protected]>
    @christianazinn @compilade
    christianazinn and compilade committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    e9895d2 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2024

  1. Merge branch 'master' into convert-split

    @christianazinn
    christianazinn committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    4e4e376 View commit details
    Browse the repository at this point in the history