2023-03-16 19:54:34 -07:00
|
|
|
[tool.poetry]
|
|
|
|
|
name = "semantic-kernel"
|
2024-07-23 10:50:38 -04:00
|
|
|
version = "1.3.0"
|
2023-12-14 08:38:36 -08:00
|
|
|
description = "Semantic Kernel Python SDK"
|
2023-03-16 19:54:34 -07:00
|
|
|
authors = ["Microsoft <SK-Support@microsoft.com>"]
|
2023-05-08 00:39:38 -07:00
|
|
|
readme = "pip/README.md"
|
2023-03-16 19:54:34 -07:00
|
|
|
packages = [{include = "semantic_kernel"}]
|
|
|
|
|
|
|
|
|
|
[tool.poetry.dependencies]
|
2024-04-22 13:42:49 +02:00
|
|
|
python = "^3.10,<3.13"
|
2024-06-29 01:03:07 +02:00
|
|
|
|
|
|
|
|
# main dependencies
|
2023-11-28 22:44:10 -08:00
|
|
|
aiohttp = "^3.8"
|
2024-06-29 01:03:07 +02:00
|
|
|
pydantic = "^2"
|
|
|
|
|
pydantic-settings = "^2"
|
|
|
|
|
defusedxml = "^0.7.1"
|
|
|
|
|
|
|
|
|
|
# embeddings
|
2024-03-12 10:27:55 +01:00
|
|
|
numpy = [
|
2024-05-22 16:46:12 +02:00
|
|
|
{ version = ">=1.25", python = "<3.12" },
|
2024-03-12 10:27:55 +01:00
|
|
|
{ version = ">=1.26", python = ">=3.12" },
|
|
|
|
|
]
|
2024-06-29 01:03:07 +02:00
|
|
|
|
|
|
|
|
# openai connector
|
2023-11-28 22:44:10 -08:00
|
|
|
openai = ">=1.0"
|
2024-06-29 01:03:07 +02:00
|
|
|
|
|
|
|
|
# openapi and swagger
|
2024-03-13 21:31:41 +00:00
|
|
|
openapi_core = ">=0.18,<0.20"
|
2024-07-25 13:18:41 -07:00
|
|
|
|
|
|
|
|
# OpenTelemetry
|
|
|
|
|
opentelemetry-api = "^1.24.0"
|
|
|
|
|
opentelemetry-sdk = "^1.24.0"
|
|
|
|
|
|
Python: Import OpenAPI documents into the semantic kernel (#2297)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
This allows us to import OpenAPI documents, including ChatGPT plugins,
into the Semantic Kernel.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
- The interface reads the operationIds of the openapi spec into a skill:
```python
from semantic_kernel.connectors.openapi import register_openapi_skill
skill = register_openapi_skill(kernel=kernel, skill_name="test", openapi_document="url/or/path/to/openapi.yamlorjson")
skill['operationId'].invoke_async()
```
- Parse an OpenAPI document
- For each operation in the document, create a function that will
execute the operation
- Add all those operations to a skill in the kernel
- Modified `import_skill` to accept a dictionary of functions instead of
just class so that we can import dynamically created functions
- Created unit tests
TESTING:
I've been testing this with the following ChatGPT plugins:
- [Semantic Kernel Starter's Python Flask
plugin](https://github.com/microsoft/semantic-kernel-starters/tree/main/sk-python-flask)
- [ChatGPT's example retrieval
plugin](https://github.com/openai/chatgpt-retrieval-plugin/blob/main/docs/providers/azuresearch/setup.md)
- This one was annoying to setup. I didn't get the plugin functioning,
but I was able to send the right API requests
- Also, their openapi file was invalid. The "servers" attribute is
misindented
- [Google ChatGPT
plugin](https://github.com/Sogody/google-chatgpt-plugin)
- [Chat TODO plugin](https://github.com/lencx/chat-todo-plugin)
- This openapi file is also invalid. I checked with an online validator.
I had to remove"required" from the referenced request objects'
properties:
https://github.com/lencx/chat-todo-plugin/blob/main/openapi.yaml#L85
Then I used this python file to test the examples:
```python
import asyncio
import logging
import semantic_kernel as sk
from semantic_kernel import ContextVariables, Kernel
from semantic_kernel.connectors.ai.open_ai import AzureTextCompletion
from semantic_kernel.connectors.openapi.sk_openapi import register_openapi_skill
# Example usage
chatgpt_retrieval_plugin = {
"openapi": # location of the plugin's openapi.yaml file,
"payload": {
"queries": [
{
"query": "string",
"filter": {
"document_id": "string",
"source": "email",
"source_id": "string",
"author": "string",
"start_date": "string",
"end_date": "string",
},
"top_k": 3,
}
]
},
"operation_id": "query_query_post",
}
sk_python_flask = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"skill_name": "FunSkill", "function_name": "Joke"},
"payload": {"input": "dinosaurs"},
"operation_id": "executeFunction",
}
google_chatgpt_plugin = {
"openapi": # location of the plugin's openapi.yaml file,
"query_params": {"q": "dinosaurs"},
"operation_id": "searchGet",
}
todo_plugin_add = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"username": "markkarle"},
"payload": {"todo": "finish this"},
"operation_id": "addTodo",
}
todo_plugin_get = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"username": "markkarle"},
"operation_id": "getTodos",
}
todo_plugin_delete = {
"openapi": # location of the plugin's openapi.yaml file,
"path_params": {"username": "markkarle"},
"payload": {"todo_idx": 0},
"operation_id": "deleteTodo",
}
plugin = todo_plugin_get # set this to the plugin you want to try
logger = logging.getLogger(__name__)
logger.addHandler(logging.StreamHandler())
logger.setLevel(logging.DEBUG)
kernel = Kernel(log=logger)
deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
kernel.add_text_completion_service(
"dv", AzureTextCompletion(deployment, endpoint, api_key)
)
skill = register_openapi_skill(
kernel=kernel, skill_name="test", openapi_document=plugin["openapi"]
)
context_variables = ContextVariables(variables=plugin)
result = asyncio.run(
skill[plugin["operation_id"]].invoke_async(variables=context_variables)
)
print(result)
```
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone :smile:
---------
Co-authored-by: Abby Harrison <abharris@microsoft.com>
2023-08-04 10:48:21 -07:00
|
|
|
prance = "^23.6.21.0"
|
2024-06-29 01:03:07 +02:00
|
|
|
|
|
|
|
|
# templating
|
2024-03-14 15:00:05 +01:00
|
|
|
pybars4 = "^0.9.13"
|
2024-03-15 09:23:25 -04:00
|
|
|
jinja2 = "^3.1.3"
|
Python: Enhanced pre commit and tasks (#5512)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
To further make the SK python robust and make it easier to get
high-quality PRs, some work on the pre-commit-config.yaml and adding a
mypy settings ini.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
Added two steps to the pre-commit-config, the first for pypy, the second
for tests with coverage.
Over time we want to mandate these checks to run against a PR before it
goes to github, that will also reduce the number of ruff and black fix
commits, and non-passing unit tests.
Added a mypy.ini, currently all but the root folder is excluded, so that
we can gradually introduce mypy coverage, did the first pieces in
kernel.py, including switch to new style annotations (using from
`__future__ import annotations`) in Kernel.
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-19 21:20:09 +01:00
|
|
|
nest-asyncio = "^1.6.0"
|
2023-03-16 19:54:34 -07:00
|
|
|
|
2024-06-29 01:03:07 +02:00
|
|
|
### Optional dependencies
|
|
|
|
|
# azure
|
|
|
|
|
azure-ai-inference = {version = "^1.0.0b1", allow-prereleases = true, optional = true}
|
|
|
|
|
azure-search-documents = {version = "11.6.0b4", allow-prereleases = true, optional = true}
|
|
|
|
|
azure-core = { version = "^1.28.0", optional = true}
|
|
|
|
|
azure-identity = { version = "^1.13.0", optional = true}
|
|
|
|
|
azure-cosmos = { version = "^4.7.0", optional = true}
|
|
|
|
|
# chroma
|
|
|
|
|
chromadb = { version = ">=0.4.13,<0.6.0", optional = true}
|
2024-07-24 16:20:36 -07:00
|
|
|
# google
|
2024-07-31 09:04:10 -07:00
|
|
|
google-cloud-aiplatform = { version = "^1.60.0", optional = true}
|
2024-07-24 16:20:36 -07:00
|
|
|
google-generativeai = { version = "^0.7.2", optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
# hugging face
|
|
|
|
|
transformers = { version = "^4.28.1", extras=["torch"], optional = true}
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
sentence-transformers = { version = "^2.2.2", optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
# mongo
|
|
|
|
|
motor = { version = "^3.3.2", optional = true }
|
|
|
|
|
# notebooks
|
|
|
|
|
ipykernel = { version = "^6.21.1", optional = true}
|
|
|
|
|
# milvus
|
2024-06-13 15:36:19 -04:00
|
|
|
pymilvus = { version = ">=2.3,<2.4.4", optional = true}
|
2024-05-22 16:46:12 +02:00
|
|
|
milvus = { version = ">=2.3,<2.3.8", markers = 'sys_platform != "win32"', optional = true}
|
2024-07-04 14:39:49 +02:00
|
|
|
# mistralai
|
|
|
|
|
mistralai = { version = "^0.4.1", optional = true}
|
2024-07-16 15:38:21 -07:00
|
|
|
# ollama
|
|
|
|
|
ollama = { version = "^0.2.1", optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
# pinecone
|
2024-05-08 10:44:34 -07:00
|
|
|
pinecone-client = { version = ">=3.0.0", optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
# postgres
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
psycopg = { version="^3.1.9", extras=["binary","pool"], optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
# qdrant
|
|
|
|
|
qdrant-client = { version = '^1.9', optional = true}
|
|
|
|
|
# redis
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
redis = { version = "^4.6.0", optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
# usearch
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
usearch = { version = "^2.9", optional = true}
|
2024-08-01 09:25:52 -04:00
|
|
|
pyarrow = { version = ">=12.0.1,<18.0.0", optional = true}
|
2024-06-29 01:03:07 +02:00
|
|
|
weaviate-client = { version = ">=3.18,<5.0", optional = true}
|
2024-07-19 10:09:23 -04:00
|
|
|
ruff = "0.5.2"
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
|
2023-03-16 19:54:34 -07:00
|
|
|
[tool.poetry.group.dev.dependencies]
|
2024-05-28 17:28:04 +02:00
|
|
|
pre-commit = ">=3.7.1"
|
|
|
|
|
ruff = ">=0.4.5"
|
|
|
|
|
ipykernel = "^6.29.4"
|
2024-05-29 17:50:55 +02:00
|
|
|
nbconvert = "^7.16.4"
|
2024-05-28 17:28:04 +02:00
|
|
|
pytest = "^8.2.1"
|
2024-08-01 14:18:10 +02:00
|
|
|
pytest-xdist = { version="^3.6.1", extras=["psutil"]}
|
|
|
|
|
pytest-cov = ">=5.0.0"
|
2024-05-28 17:28:04 +02:00
|
|
|
pytest-asyncio = "^0.23.7"
|
2024-03-12 10:27:55 +01:00
|
|
|
snoop = "^0.4.3"
|
2024-05-28 17:28:04 +02:00
|
|
|
mypy = ">=1.10.0"
|
2024-03-28 16:10:28 +01:00
|
|
|
types-PyYAML = "^6.0.12.20240311"
|
2023-03-16 19:54:34 -07:00
|
|
|
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
[tool.poetry.group.unit-tests]
|
|
|
|
|
optional = true
|
Python: Add Google PaLM connector with text completion and example file (#2076)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Implementation of Google PaLM connector with text completion and an
example file to demonstrate its functionality.
Closes #1979
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Implemented Google Palm connector with text completion
2. Added example file to ```python/samples/kernel-syntax-examples```
3. Added integration tests with different inputs to kernel.run_async
4. Added unit tests to ensure successful initialization of the class and
successful API calls
5. 3 optional arguments (top_k, safety_settings, client) for
google.generativeai.generate_text were not included. See more
information about the function and its arguments:
https://developers.generativeai.google/api/python/google/generativeai/generate_text
I also opened a PR for text embedding and chat completion #2258
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
Currently no warnings, there was 1 warning when first installing genai
with `poetry add google.generativeai==v0.1.0rc2` from within poetry
shell: "The locked version 0.1.0rc2 for google-generativeai is a yanked
version. Reason for being yanked: Release is marked as supporting Py3.8,
but in practice it requires 3.9". We would need to require later
versions of python to fix it.
---------
Co-authored-by: Abby Harrison <54643756+awharrison-28@users.noreply.github.com>
Co-authored-by: Abby Harrison <abby.harrison@microsoft.com>
2023-08-17 14:56:26 -07:00
|
|
|
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
[tool.poetry.group.unit-tests.dependencies]
|
2024-06-24 15:00:57 -07:00
|
|
|
azure-ai-inference = {version = "^1.0.0b1", allow-prereleases = true}
|
2024-05-22 16:46:12 +02:00
|
|
|
azure-search-documents = {version = "11.6.0b4", allow-prereleases = true}
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
azure-core = "^1.28.0"
|
2024-05-18 05:32:30 -07:00
|
|
|
azure-cosmos = "^4.7.0"
|
2024-07-04 14:39:49 +02:00
|
|
|
mistralai = "^0.4.1"
|
2024-07-16 15:38:21 -07:00
|
|
|
ollama = "^0.2.1"
|
2024-07-31 09:04:10 -07:00
|
|
|
google-cloud-aiplatform = "^1.60.0"
|
2024-07-24 16:20:36 -07:00
|
|
|
google-generativeai = "^0.7.2"
|
2024-06-29 01:03:07 +02:00
|
|
|
transformers = { version = "^4.28.1", extras=["torch"]}
|
2023-05-05 20:35:13 -07:00
|
|
|
sentence-transformers = "^2.2.2"
|
|
|
|
|
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
[tool.poetry.group.tests]
|
|
|
|
|
optional = true
|
2023-07-26 13:07:48 -07:00
|
|
|
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
[tool.poetry.group.tests.dependencies]
|
2024-06-29 01:03:07 +02:00
|
|
|
# azure
|
|
|
|
|
azure-ai-inference = {version = "^1.0.0b1", allow-prereleases = true}
|
|
|
|
|
azure-search-documents = {version = "11.6.0b4", allow-prereleases = true}
|
|
|
|
|
azure-core = "^1.28.0"
|
|
|
|
|
azure-identity = "^1.13.0"
|
|
|
|
|
azure-cosmos = "^4.7.0"
|
|
|
|
|
msgraph-sdk = "^1.2.0"
|
|
|
|
|
# chroma
|
|
|
|
|
chromadb = ">=0.4.13,<0.6.0"
|
2024-07-24 16:20:36 -07:00
|
|
|
# google
|
2024-07-31 09:04:10 -07:00
|
|
|
google-cloud-aiplatform = "^1.60.0"
|
2024-07-24 16:20:36 -07:00
|
|
|
google-generativeai = "^0.7.2"
|
2024-06-29 01:03:07 +02:00
|
|
|
# hugging face
|
|
|
|
|
transformers = { version = "^4.28.1", extras=["torch"]}
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
sentence-transformers = "^2.2.2"
|
2024-06-29 01:03:07 +02:00
|
|
|
# milvus
|
2024-06-13 15:36:19 -04:00
|
|
|
pymilvus = ">=2.3,<2.4.4"
|
2024-05-22 16:46:12 +02:00
|
|
|
milvus = { version = ">=2.3,<2.3.8", markers = 'sys_platform != "win32"'}
|
2024-07-04 14:39:49 +02:00
|
|
|
# mistralai
|
|
|
|
|
mistralai = "^0.4.1"
|
2024-07-16 15:38:21 -07:00
|
|
|
# ollama
|
|
|
|
|
ollama = "^0.2.1"
|
2024-06-29 01:03:07 +02:00
|
|
|
# mongodb
|
|
|
|
|
motor = "^3.3.2"
|
|
|
|
|
# pinecone
|
2024-05-08 10:44:34 -07:00
|
|
|
pinecone-client = ">=3.0.0"
|
2024-06-29 01:03:07 +02:00
|
|
|
# postgres
|
2024-03-12 10:27:55 +01:00
|
|
|
psycopg = { version="^3.1.9", extras=["binary","pool"]}
|
2024-06-29 01:03:07 +02:00
|
|
|
# qdrant
|
|
|
|
|
qdrant-client = '^1.9'
|
|
|
|
|
# redis
|
2023-08-31 13:38:16 -07:00
|
|
|
redis = "^4.6.0"
|
2024-06-29 01:03:07 +02:00
|
|
|
# usearch
|
2024-03-11 22:10:53 +01:00
|
|
|
usearch = "^2.9"
|
2024-08-01 09:25:52 -04:00
|
|
|
pyarrow = ">=12.0.1,<18.0.0"
|
2024-06-29 01:03:07 +02:00
|
|
|
# weaviate
|
|
|
|
|
weaviate-client = ">=3.18,<5.0"
|
Python: Adding USearch memory connector (#2358)
### Motivation and Context
The integration of [USearch](https://github.com/unum-cloud/usearch) as a
memory connector to Semantic Kernel (SK).
### Description
The USearch `Index` does not natively have the ability to store
different collections, and it only stores embeddings without other
attributes like `MemoryRecord`.
The `USearchMemoryStore` class encapsulates these capabilities. It uses
the USearch `Index` to store a collection of embeddings under unique
IDs, with original collection names mapped to those IDs. Other
`MemoryRecord ` attributes are stored in a `pyarrow.Table`, which is
mapped to each collection.
It's important to note the current behavior when a user removes a record
or upserts a new one with an existing ID: the old row is not removed
from the `pyarrow.Table`. This is done for performance reasons but could
lead to the table growing in size.
By default, `USearchMemoryStore` operates as an in-memory store. To
enable persistence, you must set the persist mode with calling
appropriate `__init__ `, supplying a path to the directory for the
persist files. For each collection, two files will be created:
`{collection_name}.usearch` and `{collection_name}.parquet`. Changes
will only be dumped to the disk when `close_async` is called. Due to the
interface provided by the base class `MemoryStoreBase`, this happens
implicitly when using a context manager, or it may be called explicitly.
Since collection names are used to store files on disk, all names are
converted to lowercase.
To ensure efficient use of memory, you should call `close_async`.
---------
Co-authored-by: Abby Harrison <abby.harrison@microsoft.com>
Co-authored-by: Abby Harrison <54643756+awharrison-28@users.noreply.github.com>
Co-authored-by: Devis Lucato <dluc@users.noreply.github.com>
2023-08-24 03:18:36 +04:00
|
|
|
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
# Extras are exposed to pip, this allows a user to easily add the right dependencies to their environment
|
|
|
|
|
[tool.poetry.extras]
|
2024-07-24 16:20:36 -07:00
|
|
|
all = ["transformers", "sentence-transformers", "qdrant-client", "chromadb", "pymilvus", "milvus", "mistralai", "ollama", "google", "weaviate-client", "pinecone-client", "psycopg", "redis", "azure-ai-inference", "azure-search-documents", "azure-core", "azure-identity", "azure-cosmos", "usearch", "pyarrow", "ipykernel", "motor"]
|
2024-06-29 01:03:07 +02:00
|
|
|
|
|
|
|
|
azure = ["azure-ai-inference", "azure-search-documents", "azure-core", "azure-identity", "azure-cosmos", "msgraph-sdk"]
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
chromadb = ["chromadb"]
|
2024-07-31 09:04:10 -07:00
|
|
|
google = ["google-cloud-aiplatform", "google-generativeai"]
|
2024-06-29 01:03:07 +02:00
|
|
|
hugging_face = ["transformers", "sentence-transformers"]
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
milvus = ["pymilvus", "milvus"]
|
2024-07-04 14:39:49 +02:00
|
|
|
mistralai = ["mistralai"]
|
2024-07-16 15:38:21 -07:00
|
|
|
ollama = ["ollama"]
|
2024-06-29 01:03:07 +02:00
|
|
|
mongo = ["motor"]
|
|
|
|
|
notebooks = ["ipykernel"]
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
pinecone = ["pinecone-client"]
|
|
|
|
|
postgres = ["psycopg"]
|
2024-06-29 01:03:07 +02:00
|
|
|
qdrant = ["qdrant-client"]
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
redis = ["redis"]
|
|
|
|
|
usearch = ["usearch", "pyarrow"]
|
2024-06-29 01:03:07 +02:00
|
|
|
weaviate = ["weaviate-client"]
|
Python: Add extras and minor package updates (#5440)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
Poetry groups are not the same as pip extras, so I removed the
individual groups per service, replaced with the ones needed for unit
tests and for integration tests (all).
Then added Extras, for each connector, as well as a `all` extra. Also
added a notebooks extra, this can be used instead of the first cell in
the notebooks as `pip install -U semantic-kernel[notebooks]` closing
#4713
Made all dependencies part of the main, tagged with optional.
Closes #5322
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-03-12 16:25:24 +01:00
|
|
|
|
2024-08-01 14:18:10 +02:00
|
|
|
[tool.pytest.ini_options]
|
|
|
|
|
addopts = "-ra -q -r fEX -n logical --dist loadfile --dist worksteal"
|
|
|
|
|
|
2023-04-13 14:13:19 -07:00
|
|
|
[tool.ruff]
|
|
|
|
|
line-length = 120
|
2024-05-28 17:28:04 +02:00
|
|
|
target-version = "py310"
|
|
|
|
|
include = ["*.py", "*.pyi", "**/pyproject.toml", "*.ipynb"]
|
2023-04-13 14:13:19 -07:00
|
|
|
|
2024-05-28 17:28:04 +02:00
|
|
|
[tool.ruff.lint]
|
2024-05-31 15:06:41 +02:00
|
|
|
preview = true
|
|
|
|
|
select = ["D", "E", "F", "I", "CPY", "ISC", "INP", "RSE102", "RET", "SIM", "TD", "FIX", "ERA001", "RUF"]
|
|
|
|
|
ignore = ["D100", "D101", "D104", "TD003", "FIX002"]
|
2024-05-28 17:28:04 +02:00
|
|
|
|
|
|
|
|
[tool.ruff.lint.pydocstyle]
|
|
|
|
|
convention = "google"
|
|
|
|
|
|
|
|
|
|
[tool.ruff.lint.per-file-ignores]
|
2024-05-31 15:06:41 +02:00
|
|
|
# Ignore all directories named `tests` and `samples`.
|
|
|
|
|
"tests/**" = ["D", "INP", "TD", "ERA001", "RUF"]
|
|
|
|
|
"samples/**" = ["D", "INP", "ERA001", "RUF"]
|
2024-05-28 17:28:04 +02:00
|
|
|
# Ignore all files that end in `_test.py`.
|
|
|
|
|
"*_test.py" = ["D"]
|
2024-05-31 15:06:41 +02:00
|
|
|
"*.ipynb" = ["CPY", "E501"]
|
|
|
|
|
|
|
|
|
|
[tool.ruff.lint.flake8-copyright]
|
|
|
|
|
notice-rgx = "^# Copyright \\(c\\) Microsoft\\. All rights reserved\\."
|
|
|
|
|
min-file-size = 1
|
2024-05-28 17:28:04 +02:00
|
|
|
|
|
|
|
|
[tool.bandit]
|
|
|
|
|
targets = ["python/semantic_kernel"]
|
|
|
|
|
exclude_dirs = ["python/tests"]
|
Python: set line-length for black in sync with Ruff, run black. (#4396)
### Motivation and Context
<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
1. Why is this change required?
2. What problem does it solve?
3. What scenario does it contribute to?
4. If it fixes an open issue, please link to the issue here.
-->
As we are tightening the formatting and linting setups, this was
missing, now the line-length for black and ruff are the same, thereby no
longer running into black issues as often.
### Description
<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
### Contribution Checklist
<!-- Before submitting this PR, please make sure: -->
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile:
2024-01-04 15:28:20 +01:00
|
|
|
|
2023-03-16 19:54:34 -07:00
|
|
|
[build-system]
|
|
|
|
|
requires = ["poetry-core"]
|
|
|
|
|
build-backend = "poetry.core.masonry.api"
|
2024-05-28 17:28:04 +02:00
|
|
|
|
