abetlen/llama-cpp-python
1
10.1k
Fork 0
mirror of https://github.com/abetlen/llama-cpp-python.git synced 2026-03-26 07:21:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,004 Commits 23 Branches 349 Tags
cce48873166e6b6fbfe0d944a6184ef11858e735
Commit Graph

158 Commits

Author SHA1 Message Date
Andrei Betlen
cce4887316 fix(ci): Fix macos cpu builds 2025-07-06 01:32:07 -04:00
Andrei Betlen
98fda8cdf9 fix(ci): Temporarily disable windows cuda wheels 2025-07-06 00:55:34 -04:00
Andrei Betlen
7011bc198d fix(ci): Update docker runner 2025-07-06 00:45:59 -04:00
Andrei Betlen
82ad829c4c fix(ci): update runners for cpu builds 2025-07-05 02:35:56 -04:00
Andrei Betlen
11d28df7d8 fix(ci): Remove macos-13 builds to fix cross compilation error 2025-07-05 02:23:36 -04:00
Andrei Betlen
ae54cde83a fix(ci): Update cuda build action to use ubuntu 22.04 2025-07-05 02:20:25 -04:00
oobabooga
14879c7819 fix(ci): Fix the CUDA workflow (#1894) 2025-01-28 21:43:29 -05:00
Andrei Betlen
ea4d86ac25 fix(ci): update macos runner image to non-deprecated version 2024-12-09 00:42:15 -05:00
Andrei Betlen
ca808028bd fix(ci): hotfix for wheels 2024-12-08 23:51:49 -05:00
Andrei Betlen
61508c265a Add CUDA 12.5 and 12.6 to generated output wheels 2024-12-08 22:55:03 -05:00
Olivier DEBAUCHE
77a12a3966 fix: Re-add suport for CUDA 12.5, add CUDA 12.6 (#1775) 
Add support for Cuda 12.6.1
Update version of Cuda 12.5.0 to 12.5.1

Co-authored-by: Andrei <abetlen@gmail.com>
 2024-12-06 05:01:17 -05:00
dependabot[bot]
ddac04c78e chore(deps): bump conda-incubator/setup-miniconda from 3.0.4 to 3.1.0 (#1821) 
Bumps [conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/conda-incubator/setup-miniconda/releases)
- [Changelog](https://github.com/conda-incubator/setup-miniconda/blob/main/CHANGELOG.md)
- [Commits](https://github.com/conda-incubator/setup-miniconda/compare/v3.0.4...v3.1.0)

---
updated-dependencies:
- dependency-name: conda-incubator/setup-miniconda
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-06 04:39:27 -05:00
Andrei Betlen
d553a54333 Merge branch 'main' of github.com:abetlen/llama-cpp-python into main 2024-12-06 04:37:02 -05:00
dependabot[bot]
27953036b1 chore(deps): bump pypa/cibuildwheel from 2.21.1 to 2.22.0 (#1844) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.21.1 to 2.22.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.21.1...v2.22.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-06 04:32:37 -05:00
Andrei Betlen
991d9cde1b fix(ci): Remove CUDA 12.5 from index 2024-12-06 04:32:19 -05:00
PΛBLØ ᄃΞ
4f17ae5af0 fix(ci): Remove cuda version 12.5.0 incompatibility with VS (#1838) 
Co-authored-by: Andrei <abetlen@gmail.com>
 2024-12-06 04:31:01 -05:00
Andrei Betlen
d8cc231fd0 fix(ci): Use default architecture chosen by action 2024-12-06 04:06:07 -05:00
Andrei Betlen
b34f20046b fix(ci): Use python3 2024-12-06 03:59:51 -05:00
Andrei Betlen
1cd3f2cc6a fix(ci): gg 2024-12-06 03:52:54 -05:00
Andrei Betlen
df05096fc5 fix(ci): Install with regular pip 2024-12-06 03:42:33 -05:00
Andrei Betlen
a412ba5539 fix(ci): Update config 2024-12-06 03:38:12 -05:00
Andrei Betlen
9a09fc7848 fix(ci): Debug print python system architecture 2024-12-06 03:19:36 -05:00
Andrei Betlen
f11a781d86 fix(ci): Use macos-13 runner 2024-12-06 03:04:48 -05:00
Andrei Betlen
8988aaf7f6 fix(ci): Use macos-14 runner 2024-12-06 03:02:04 -05:00
Andrei Betlen
72ed7b8855 fix(ci): Explicitly test on arm64 macos runner 2024-12-06 02:55:57 -05:00
Andrei Betlen
9d06e36c00 fix(ci): Explicitly install arm64 python version 2024-12-06 02:49:20 -05:00
Andrei Betlen
65222bc493 fix: install build dependency 2024-09-25 14:27:19 -04:00
Andrei Betlen
8e07db0f87 fix: install build dependency 2024-09-25 14:17:04 -04:00
dependabot[bot]
1324c0c50c chore(deps): bump actions/cache from 3 to 4 (#1751) 
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrei <abetlen@gmail.com>
 2024-09-20 18:04:59 -04:00
Andrei
f8fcb3ea34 feat: Update sampling API for llama.cpp (#1742) 
* Initial samplng api update

* Fix logger

* Update tests

* Update

* Remove seed

* Add sampling chain

* Remove unnused test

* Use Qwen2 0.5B for ci tests

* Fix typo

* Fix typo

* Update cache version

* Use real model for tests

* Add huggingface-hub as a test dependency

* Remove RUST_LOG=trace

* Add actual logit processor test
 2024-09-18 20:00:19 -04:00
dependabot[bot]
a4e1451abe chore(deps): bump pypa/cibuildwheel from 2.20.0 to 2.21.1 (#1743) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.20.0 to 2.21.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.20.0...v2.21.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrei <abetlen@gmail.com>
 2024-09-18 19:26:51 -04:00
Olivier DEBAUCHE
e529940f45 feat(ci): Speed up CI workflows using uv, add support for CUDA 12.5 wheels 
* Update build-wheels-cuda.yaml

* Update build-wheels-cuda.yaml

* revert

* Bump pyhton from 3.8 to 3.9

* Remove python 3.8

* Remove Python 3.7 and 3.8 deprecated

* Bump python from 3.8 to 3.9

* Add python 3.9

* Add python 3.9, remove macos-11 deprecated, add macos-14

* Bump python 3.8 to 3.9

* Add python 3.13

* Add python 3.13

* python 3.13 remove

* remove python 3.13

* remove python 3.8

* Bump macos-13 to macos-14

* Update build-wheels-metal.yaml

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update generate-index-from-release.yaml

Add avx, avx2 and avx512

* Update test.yaml

* Update test-pypi.yaml

* Update publish.yaml

* Update publish-to-test.yaml

* Update build-wheels-cuda.yaml

Cuda with AVX2 by default

* Update build-wheels-cuda.yaml

* remove DEPRECATED 32 bits

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

Upgrade matrix os to latest version

* Update build-wheels-metal.yaml

* Update build-wheels-cuda.yaml

* Update test.yaml

* Update test-pypi.yaml

* Update test.yaml

Add cache: 'pip'

* Update publish-to-test.yaml

* Update build-wheels-metal.yaml

Add cache: 'pip'

* Update build-wheels-cuda.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

remove x86_64

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* revert

* Remove cpu variants

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update publish-to-test.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update publish.yaml

* Update test-pypi.yaml

* Update test.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update publish.yaml

* Update test-pypi.yaml

* Update publish-to-test.yaml

* Update test.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update publish-to-test.yaml

* Update publish.yaml

* Update test-pypi.yaml

* Update test.yaml

* Update test.yaml

* Update build-and-release.yaml

* Update publish-to-test.yaml

* Update build-wheels-metal.yaml

* Update test-pypi.yaml

* Update test.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update build-wheels-metal.yaml

* Update publish.yaml

* Update publish-to-test.yaml

* Update test-pypi.yaml

* Update test.yaml

* Update build-wheels-cuda.yaml

* Update generate-index-from-release.yaml

* Update README.md

* Update README.md

* Update test.yaml

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>
 2024-09-18 19:22:05 -04:00
Andrei Betlen
c68e7fbedf fix: pull all gh releases for self-hosted python index 2024-08-29 01:51:25 -04:00
dependabot[bot]
131db40770 chore(deps): bump pypa/cibuildwheel from 2.19.2 to 2.20.0 (#1657) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.19.2 to 2.20.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.19.2...v2.20.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-08-07 10:42:27 -04:00
Andrei Betlen
198f47dc1b feat(ci): Re-build wheel index automatically when releases are created 2024-08-07 09:33:05 -04:00
dependabot[bot]
f95057a475 chore(deps): bump microsoft/setup-msbuild from 1.3 to 2 (#1585) 
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.3 to 2.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](https://github.com/microsoft/setup-msbuild/compare/v1.3...v2)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-07-20 17:16:13 -04:00
dependabot[bot]
e31f0968ff chore(deps): bump microsoft/setup-msbuild from 1.1 to 1.3 (#1569) 
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 1.1 to 1.3.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Changelog](https://github.com/microsoft/setup-msbuild/blob/main/building-release.md)
- [Commits](https://github.com/microsoft/setup-msbuild/compare/v1.1...v1.3)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrei <abetlen@gmail.com>
 2024-07-09 11:51:46 -04:00
dependabot[bot]
14760c63ed chore(deps): bump pypa/cibuildwheel from 2.19.1 to 2.19.2 (#1568) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.19.1 to 2.19.2.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.19.1...v2.19.2)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrei <abetlen@gmail.com>
 2024-07-09 11:51:35 -04:00
Andrei Betlen
09a4f78e05 fix(ci): Update LLAMA_ flags to GGML_ 2024-07-09 00:37:03 -04:00
Andrei Betlen
3a551eb526 fix(ci): Update macos image (macos-11 is removed) 2024-07-02 11:58:26 -04:00
Andrei Betlen
4fb6fc12a0 fix(ci): Use LLAMA_CUDA for cuda wheels 2024-07-02 03:09:19 -04:00
Andrei Betlen
dc20e8c342 fix: Copy dependencies for windows 
fix: :(

fix

fix

fix: Copy runtime dlls on windows

fix: Add explicit copy command for windows dlls

fix

fix

fix: >:(

fix

fix

fix

fix

Update path on windows

check dll dependancies

fix: Update PATH on win32

ci: Update test.yaml
 2024-07-02 01:39:03 -04:00
oobabooga
73ddf297be fix(ci): Fix the CUDA workflow (#1551) 2024-07-01 21:31:25 -04:00
dependabot[bot]
35c980eb2e chore(deps): bump pypa/cibuildwheel from 2.18.1 to 2.19.1 (#1527) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.18.1 to 2.19.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.18.1...v2.19.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-06-21 12:10:43 -04:00
dependabot[bot]
398fe81547 chore(deps): bump docker/build-push-action from 5 to 6 (#1539) 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-06-21 12:10:34 -04:00
Olivier DEBAUCHE
9e396b3ebd feat: Update workflows and pre-built wheels (#1416) 
* Update build-wheels-cuda.yaml

* Update build-wheels-cuda.yaml

* revert

* Bump pyhton from 3.8 to 3.9

* Remove python 3.8

* Remove Python 3.7 and 3.8 deprecated

* Bump python from 3.8 to 3.9

* Add python 3.9

* Add python 3.9, remove macos-11 deprecated, add macos-14

* Bump python 3.8 to 3.9

* Add python 3.13

* Add python 3.13

* python 3.13 remove

* remove python 3.13

* remove python 3.8

* Bump macos-13 to macos-14

* Update build-wheels-metal.yaml

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update generate-index-from-release.yaml

Add avx, avx2 and avx512

* Update test.yaml

* Update test-pypi.yaml

* Update publish.yaml

* Update publish-to-test.yaml

* Update build-wheels-cuda.yaml

Cuda with AVX2 by default

* Update build-wheels-cuda.yaml

* remove DEPRECATED 32 bits

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

Upgrade matrix os to latest version

* Update build-wheels-metal.yaml

* Update build-wheels-cuda.yaml

* Update test.yaml

* Update test-pypi.yaml

* Update test.yaml

Add cache: 'pip'

* Update publish-to-test.yaml

* Update build-wheels-metal.yaml

Add cache: 'pip'

* Update build-wheels-cuda.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

remove x86_64

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-and-release.yaml

* Update build-wheels-metal.yaml

* revert

* Remove cpu variants

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>
 2024-06-13 10:19:57 -04:00
dependabot[bot]
5af81634cb chore(deps): bump pypa/cibuildwheel from 2.18.1 to 2.19.0 (#1522) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.18.1 to 2.19.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.18.1...v2.19.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-06-13 10:12:02 -04:00
dependabot[bot]
c564007ff6 chore(deps): bump pypa/cibuildwheel from 2.18.0 to 2.18.1 (#1472) 
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrei <abetlen@gmail.com>
 2024-05-27 10:57:17 -04:00
dependabot[bot]
4b54f79330 chore(deps): bump pypa/cibuildwheel from 2.17.0 to 2.18.0 (#1453) 
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 2.17.0 to 2.18.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v2.17.0...v2.18.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-14 09:35:52 -04:00
Olivier DEBAUCHE
3666833107 feat(ci): Add docker checks and check deps more frequently (#1426) 
* Update dependabot.yml

Add github-actions update

* Update dependabot.yml

* Update dependabot.yml
 2024-05-05 12:42:28 -04:00