Morph MORPH
SIGN IN SIGN UP
microsoft / semantic-kernel UNCLAIMED

Integrate cutting-edge LLM technology quickly and easily into your apps

0 0 24 C#
Normal View History Raw
Python: Text partitioning module (#450) ### Motivation and Context Porting the text partitioning module to python (Reopening of PR #427 ) ### Description - This is adding the Text partitioning module in `semantic_kernel/semantic_functions/semantic_text_partitioner.py` and the function_extention in `semantic_kernel/semantic_functions/function_extension.py` - Compared to the C# version the files were added directly into the `semantic_functions` directory instead of the `semantic_functions/partitioning` to not have too many nested directories.
2023-04-26 18:50:24 -04:00
# Copyright (c) Microsoft. All rights reserved.
Python: major features for the Kernel Arguments, Function Result and new Prompt Templating (#5077) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Major work to replace the context with Kernel Arguments on the front-end and Function Results on the back. Updated the function decorator to a new approach, in line with dotnet. Revamps the way function are called, allowing native functions to be completely ignorant of SK, other then the decorator. This also moves things into the new folder structure in sync with dotnet. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds: - KernelArguments, a dict-like class that replaces the variables in KernelContext. Closes #4565 - FunctionResult, a class to hold the results of function executions, includes the function, the value and metadata, as well as two convenient function to get the value out of it (str and get_inner_content) the first is generic, the second specifically for KernelContent responses (from AI services). - AI Service Selector class, has a default, which is based on the order in the arguments followed by the order in the functions, can be overridden to implement your own strategy. Closes #4631 - Introduces ChatHistory and refactors the PromptTemplateConfig. Closes #4856, #4630 - Improves parsing of templates, will now all validate during creation and throw an error then, instead of some that do not check for validaty until used. - Introduces named_args block and thereby the ability to have multiple arguments for a function call in a template. Closes #5003 Updates: - kernel_function decorators, the parameter decorator was removed and instead we now use Annotated to add a description of a field and we get the type and required from the function definition. - core plugins, use the new approach to kernel_function decorators. - planners, template engines have all been updated to use the kernel and kernelarguments instead of Context. - Events have been updated, now use kernelarguments and function_result - Tokenizers support for named_args and improvements on parsing and checking. - Kernel examples and notebooks to use the latest code. - All unit and integration tests. There is more code coverage now than before. Removed: - kernelContext - kernel_function_parameter_decorator - delegate handling code for native functions - file_io_plugin and tests - SemanticFunctionConfig - ChatPromptTemplate ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone :smile: --------- Co-authored-by: Evan Mattson <evmattso@microsoft.com> Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com> Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
2024-02-24 21:05:40 +01:00
from semantic_kernel.functions.kernel_arguments import KernelArguments
from semantic_kernel.functions.kernel_function import KernelFunction
from semantic_kernel.kernel import Kernel
Python: Text partitioning module (#450) ### Motivation and Context Porting the text partitioning module to python (Reopening of PR #427 ) ### Description - This is adding the Text partitioning module in `semantic_kernel/semantic_functions/semantic_text_partitioner.py` and the function_extention in `semantic_kernel/semantic_functions/function_extension.py` - Compared to the C# version the files were added directly into the `semantic_functions` directory instead of the `semantic_functions/partitioning` to not have too many nested directories.
2023-04-26 18:50:24 -04:00
Python: Removes the _async suffix where is not needed (#4735) ### Motivation and Context Delete all *_async where *_sync doesn't exist to make the library more Pythonic. Fixes #4629 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [X] The code builds clean without any errors or warnings - [X] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [X] All unit tests pass, and I have added new tests where possible - [X] I didn't break anyone :smile: --------- Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
2024-01-26 10:15:35 -07:00
async def aggregate_chunked_results(
Python: upgraded all files to 310 plus format and removed from future (#6353) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Now that we only support 310 and up, we could unify all files, to use the new style typing and annotations. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone :smile:
2024-05-21 18:33:08 +02:00
func: KernelFunction, chunked_results: list[str], kernel: Kernel, arguments: KernelArguments
Python: major features for the Kernel Arguments, Function Result and new Prompt Templating (#5077) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Major work to replace the context with Kernel Arguments on the front-end and Function Results on the back. Updated the function decorator to a new approach, in line with dotnet. Revamps the way function are called, allowing native functions to be completely ignorant of SK, other then the decorator. This also moves things into the new folder structure in sync with dotnet. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds: - KernelArguments, a dict-like class that replaces the variables in KernelContext. Closes #4565 - FunctionResult, a class to hold the results of function executions, includes the function, the value and metadata, as well as two convenient function to get the value out of it (str and get_inner_content) the first is generic, the second specifically for KernelContent responses (from AI services). - AI Service Selector class, has a default, which is based on the order in the arguments followed by the order in the functions, can be overridden to implement your own strategy. Closes #4631 - Introduces ChatHistory and refactors the PromptTemplateConfig. Closes #4856, #4630 - Improves parsing of templates, will now all validate during creation and throw an error then, instead of some that do not check for validaty until used. - Introduces named_args block and thereby the ability to have multiple arguments for a function call in a template. Closes #5003 Updates: - kernel_function decorators, the parameter decorator was removed and instead we now use Annotated to add a description of a field and we get the type and required from the function definition. - core plugins, use the new approach to kernel_function decorators. - planners, template engines have all been updated to use the kernel and kernelarguments instead of Context. - Events have been updated, now use kernelarguments and function_result - Tokenizers support for named_args and improvements on parsing and checking. - Kernel examples and notebooks to use the latest code. - All unit and integration tests. There is more code coverage now than before. Removed: - kernelContext - kernel_function_parameter_decorator - delegate handling code for native functions - file_io_plugin and tests - SemanticFunctionConfig - ChatPromptTemplate ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone :smile: --------- Co-authored-by: Evan Mattson <evmattso@microsoft.com> Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com> Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
2024-02-24 21:05:40 +01:00
) -> str:
Python: new pre-commit actions and pre-commit as GHA (#6376) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> - Added static code checking to pre-commit (check-ast and nbqa-check-ast) - Added Bandit security checking to pre-commit - Added pre-commit step to python-lint workflow, if it works, can delete seperate mypy, ruff and black tests ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone :smile:
2024-05-28 17:28:04 +02:00
"""Aggregate the results from the chunked results."""
Python: Text partitioning module (#450) ### Motivation and Context Porting the text partitioning module to python (Reopening of PR #427 ) ### Description - This is adding the Text partitioning module in `semantic_kernel/semantic_functions/semantic_text_partitioner.py` and the function_extention in `semantic_kernel/semantic_functions/function_extension.py` - Compared to the C# version the files were added directly into the `semantic_functions` directory instead of the `semantic_functions/partitioning` to not have too many nested directories.
2023-04-26 18:50:24 -04:00
results = []
for chunk in chunked_results:
Python: major features for the Kernel Arguments, Function Result and new Prompt Templating (#5077) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Major work to replace the context with Kernel Arguments on the front-end and Function Results on the back. Updated the function decorator to a new approach, in line with dotnet. Revamps the way function are called, allowing native functions to be completely ignorant of SK, other then the decorator. This also moves things into the new folder structure in sync with dotnet. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds: - KernelArguments, a dict-like class that replaces the variables in KernelContext. Closes #4565 - FunctionResult, a class to hold the results of function executions, includes the function, the value and metadata, as well as two convenient function to get the value out of it (str and get_inner_content) the first is generic, the second specifically for KernelContent responses (from AI services). - AI Service Selector class, has a default, which is based on the order in the arguments followed by the order in the functions, can be overridden to implement your own strategy. Closes #4631 - Introduces ChatHistory and refactors the PromptTemplateConfig. Closes #4856, #4630 - Improves parsing of templates, will now all validate during creation and throw an error then, instead of some that do not check for validaty until used. - Introduces named_args block and thereby the ability to have multiple arguments for a function call in a template. Closes #5003 Updates: - kernel_function decorators, the parameter decorator was removed and instead we now use Annotated to add a description of a field and we get the type and required from the function definition. - core plugins, use the new approach to kernel_function decorators. - planners, template engines have all been updated to use the kernel and kernelarguments instead of Context. - Events have been updated, now use kernelarguments and function_result - Tokenizers support for named_args and improvements on parsing and checking. - Kernel examples and notebooks to use the latest code. - All unit and integration tests. There is more code coverage now than before. Removed: - kernelContext - kernel_function_parameter_decorator - delegate handling code for native functions - file_io_plugin and tests - SemanticFunctionConfig - ChatPromptTemplate ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone :smile: --------- Co-authored-by: Evan Mattson <evmattso@microsoft.com> Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com> Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
2024-02-24 21:05:40 +01:00
arguments["input"] = chunk
result = await func.invoke(kernel, arguments)
Python: Text partitioning module (#450) ### Motivation and Context Porting the text partitioning module to python (Reopening of PR #427 ) ### Description - This is adding the Text partitioning module in `semantic_kernel/semantic_functions/semantic_text_partitioner.py` and the function_extention in `semantic_kernel/semantic_functions/function_extension.py` - Compared to the C# version the files were added directly into the `semantic_functions` directory instead of the `semantic_functions/partitioning` to not have too many nested directories.
2023-04-26 18:50:24 -04:00
Python: major features for the Kernel Arguments, Function Result and new Prompt Templating (#5077) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Major work to replace the context with Kernel Arguments on the front-end and Function Results on the back. Updated the function decorator to a new approach, in line with dotnet. Revamps the way function are called, allowing native functions to be completely ignorant of SK, other then the decorator. This also moves things into the new folder structure in sync with dotnet. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds: - KernelArguments, a dict-like class that replaces the variables in KernelContext. Closes #4565 - FunctionResult, a class to hold the results of function executions, includes the function, the value and metadata, as well as two convenient function to get the value out of it (str and get_inner_content) the first is generic, the second specifically for KernelContent responses (from AI services). - AI Service Selector class, has a default, which is based on the order in the arguments followed by the order in the functions, can be overridden to implement your own strategy. Closes #4631 - Introduces ChatHistory and refactors the PromptTemplateConfig. Closes #4856, #4630 - Improves parsing of templates, will now all validate during creation and throw an error then, instead of some that do not check for validaty until used. - Introduces named_args block and thereby the ability to have multiple arguments for a function call in a template. Closes #5003 Updates: - kernel_function decorators, the parameter decorator was removed and instead we now use Annotated to add a description of a field and we get the type and required from the function definition. - core plugins, use the new approach to kernel_function decorators. - planners, template engines have all been updated to use the kernel and kernelarguments instead of Context. - Events have been updated, now use kernelarguments and function_result - Tokenizers support for named_args and improvements on parsing and checking. - Kernel examples and notebooks to use the latest code. - All unit and integration tests. There is more code coverage now than before. Removed: - kernelContext - kernel_function_parameter_decorator - delegate handling code for native functions - file_io_plugin and tests - SemanticFunctionConfig - ChatPromptTemplate ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone :smile: --------- Co-authored-by: Evan Mattson <evmattso@microsoft.com> Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com> Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
2024-02-24 21:05:40 +01:00
results.append(str(result))
Python: Text partitioning module (#450) ### Motivation and Context Porting the text partitioning module to python (Reopening of PR #427 ) ### Description - This is adding the Text partitioning module in `semantic_kernel/semantic_functions/semantic_text_partitioner.py` and the function_extention in `semantic_kernel/semantic_functions/function_extension.py` - Compared to the C# version the files were added directly into the `semantic_functions` directory instead of the `semantic_functions/partitioning` to not have too many nested directories.
2023-04-26 18:50:24 -04:00
Python: major features for the Kernel Arguments, Function Result and new Prompt Templating (#5077) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Major work to replace the context with Kernel Arguments on the front-end and Function Results on the back. Updated the function decorator to a new approach, in line with dotnet. Revamps the way function are called, allowing native functions to be completely ignorant of SK, other then the decorator. This also moves things into the new folder structure in sync with dotnet. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> Adds: - KernelArguments, a dict-like class that replaces the variables in KernelContext. Closes #4565 - FunctionResult, a class to hold the results of function executions, includes the function, the value and metadata, as well as two convenient function to get the value out of it (str and get_inner_content) the first is generic, the second specifically for KernelContent responses (from AI services). - AI Service Selector class, has a default, which is based on the order in the arguments followed by the order in the functions, can be overridden to implement your own strategy. Closes #4631 - Introduces ChatHistory and refactors the PromptTemplateConfig. Closes #4856, #4630 - Improves parsing of templates, will now all validate during creation and throw an error then, instead of some that do not check for validaty until used. - Introduces named_args block and thereby the ability to have multiple arguments for a function call in a template. Closes #5003 Updates: - kernel_function decorators, the parameter decorator was removed and instead we now use Annotated to add a description of a field and we get the type and required from the function definition. - core plugins, use the new approach to kernel_function decorators. - planners, template engines have all been updated to use the kernel and kernelarguments instead of Context. - Events have been updated, now use kernelarguments and function_result - Tokenizers support for named_args and improvements on parsing and checking. - Kernel examples and notebooks to use the latest code. - All unit and integration tests. There is more code coverage now than before. Removed: - kernelContext - kernel_function_parameter_decorator - delegate handling code for native functions - file_io_plugin and tests - SemanticFunctionConfig - ChatPromptTemplate ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone :smile: --------- Co-authored-by: Evan Mattson <evmattso@microsoft.com> Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com> Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
2024-02-24 21:05:40 +01:00
return "\n".join(results)
Reference in New Issue Copy Permalink