Files
langflow/.gitattributes
Sebastián Estévez 5493ac7b0b feat: voice mode (#4642)
* WIP

* works

* stereo

* ui v0

* unnecessary import

* update steps in voice ws

* [autofix.ci] apply automated fixes

* unused

* merge

* [autofix.ci] apply automated fixes

* cleanly handle missing OPENAI key

* ruff

* [autofix.ci] apply automated fixes

* fix genericIconComponent path

* update for recent async fixes

* accidentally commited html file

* better prompt and threading

* client barge-in detection

* fmt

* error handling

* fixed VAD with 24 to 16Hz resampling

* comment out debug file

* better_vad

* lock

* router

* [autofix.ci] apply automated fixes

* mcp fixes

* timeout fix

* global variable exception handling

* don't close the websocket

* fix double send bug

* fix double send bug

* response.output_item event type typo

* voice_mode logging

* 📝 (constants.py): Add "copy_field" attribute to FIELD_FORMAT_ATTRIBUTES list
📝 (webhook.py): Add "copy_field" attribute to MultilineInput component
📝 (input_mixin.py): Add "copy_field" attribute to BaseInputMixin class
📝 (inputs.py): Add "copy_field" attribute to StrInput class
📝 (template/field/base.py): Add "copy_field" attribute to Input class
🚀 (NodeDescription/index.tsx): Remove default placeholder text for emptyPlaceholder prop
 (copyFieldAreaComponent/index.tsx): Add new component for handling copy field functionality
♻️ (strRenderComponent/index.tsx): Refactor component to include CopyFieldAreaComponent when copy_field attribute is present in template data

*  (NodeDescription/index.tsx): refactor renderedDescription useMemo to improve readability and maintainability
♻️ (GenericNode/index.tsx): refactor code to improve readability and maintainability, and optimize rendering logic

* 📝 (webhook.py): Add cURL field to WebhookComponent for better integration with external systems
📝 (graph/base.py): Add logging of vertex build information in Graph class for debugging purposes
📝 (NodeInputField/index.tsx): Add nodeInformationMetadata to NodeInputField for better tracking of node information
📝 (copyFieldAreaComponent/index.tsx): Refactor CopyFieldAreaComponent to handle different types of values, including webhooks
📝 (strRenderComponent/index.tsx): Add WebhookFieldComponent to handle webhook type in StrRenderComponent
📝 (tableNodeCellRender/index.tsx): Add nodeInformationMetadata to TableNodeCellRender for better tracking of node information
📝 (textAreaComponent/index.tsx): Add support for webhook format in TextAreaComponent for better integration with webhooks
📝 (webhookFieldComponent/index.tsx): Add WebhookFieldComponent to handle webhook type in ParameterRenderComponent
📝 (custom-parameter.tsx): Add nodeInformationMetadata to CustomParameterComponent for better tracking of node information
📝 (get-curl-code.tsx): Add support for different formats in getCurlWebhookCode for generating cURL commands
📝 (textAreaModal/index.tsx): Add onCloseModal callback to ComponentTextModal for better handling of modal closing
📝 (index.ts): Add type field to APIClassType for better typing of API classes

*  (index.tsx): Add a button to generate a token in the WebhookFieldComponent for improved user experience and functionality. Update the structure of the component to include the new button and styling adjustments.

* [autofix.ci] apply automated fixes

*  (generate-token-dialog.tsx): add GenerateTokenDialog component to handle token generation in webhookFieldComponent
📝 (index.tsx): import and use GenerateTokenDialog component in WebhookFieldComponent for token generation functionality

*  (frontend): introduce new feature to create API keys with customizable modal properties
🔧 (frontend): add modalProps object to customize modal title, description, input label, input placeholder, button text, generated key message, and show icon flag

* add pool interval variable and tests

* 📝 (NodeOutputfield): Remove unused ScanEyeIcon component
 (validate-webhook.ts): Add function to validate webhook data before processing
♻️ (use-get-builds-pooling-mutation): Refactor to set flow pool based on current flow
🔧 (content-render.tsx): Add data-testid attribute to api key input element
🔧 (webhookComponent.spec.ts): Refactor test to use waitForRequest for monitoring build requests

* [autofix.ci] apply automated fixes

* 🔧 (backend): rename webhook_pooling_interval to webhook_polling_interval for consistency
🔧 (frontend): update references to webhook_pooling_interval to webhook_polling_interval for consistency

* vad + dummy check

* 📝 (frontend): Update import paths and remove unused imports for better code organization and maintainability
🔧 (frontend): Refactor background styles in components to use constants for consistency and easier theming
🚀 (frontend): Add custom SecretKeyModalButton component for better modularity and reusability

* 📝 (use-get-api-keys.ts): add a TODO comment to request API key from DSLF endpoint for future implementation.

* 📝 (input_mixin.py): Remove copy_field attribute from BaseInputMixin as it is no longer needed
♻️ (inputs.py): Remove copy_field attribute from StrInput class as it is no longer needed
♻️ (inputs.py): Set copy_field attribute to False in MultilineInput class to ensure consistency
♻️ (template/field/base.py): Remove copy_field attribute from Input class as it is no longer needed
📝 (textAreaComponent/index.tsx): Replace hardcoded value "CURL_WEBHOOK" with constant WEBHOOK_VALUE for better readability and maintainability

* 🐛 (base.py): fix issue where flow_id could be None by defaulting to an empty string if flow_id is None

* 🔧 (secret-key-modal.tsx): Remove unused SecretKeyModalButton component
🔧 (get-modal-props.tsx): Remove unused getModalPropsApiKey function and related imports and constants

* 📝 (langflow): add noqa comments to suppress linting rule A005 for specific files in the io, logging, and socket modules

*  (frontend): Add voice assistant feature to chat input component
🔧 (frontend): Refactor import path for VoiceAssistant component
🔧 (frontend): Refactor class name for button in upload-file-button component
🔧 (frontend): Refactor class name for button in voice-button component
🔧 (frontend): Refactor class name for button in applies.css
🔧 (frontend): Refactor class name for button in styleUtils.ts

*  (api-key-popup.tsx): Add a new component ApiKeyPopup to enable users to enter their OpenAI API key for voice transcription
♻️ (index.tsx): Refactor VoiceAssistant component to check for the presence of OpenAI API key before starting voice transcription and show ApiKeyPopup component if key is missing
🔧 (apiKeyModal/index.tsx): Remove the obsolete APIKeyModal component as it is no longer needed after implementing ApiKeyPopup in the voice assistant feature

* merge fix

*  (Voice Assistant): Introduce voice transcription feature powered by OpenAI. Add components and hooks for handling audio recording, processing, and WebSocket communication. Implement functionality to start, stop recording, play audio chunks, handle WebSocket messages, and initialize audio context. Add support for entering API key for OpenAI.

*  (voice-assistant.tsx): Add voice assistant feature to the chat input component for recording and processing audio input
🔧 (use-post-voice.tsx): Remove unused file use-post-voice.tsx from the project
♻️ (use-handle-websocket-message.ts): Refactor useHandleWebsocketMessage function to improve readability and remove unnecessary console logs
♻️ (use-initialize-audio.ts): Refactor useInitializeAudio function to handle audio context creation and resume more efficiently
 (use-interrupt-playback.ts): Add useInterruptPlayback function to handle interrupting audio playback
 (use-start-conversation.ts): Add useStartConversation function to initiate a conversation using a WebSocket connection
📝 (chat-input.tsx): Update import path for VoiceAssistant component to match the new file structure

* ♻️ (use-interrupt-playback.ts): refactor useInterruptPlayback hook to clear audio queue, stop playback, and send stop message to audio processor if it exists

* 🔧 (gitattributes): add *.raw file extension as binary to prevent git from modifying the file contents

* lazy load components

* TRACE logging

* new voice_mode ws endpoint and elevenlabs mode

* unique voice

* text modality

* stream elevenlabs

* ws and sentence chunking

* offload tts to new thread

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* add eleven labs changes on FE

* 🐛 (use-handle-websocket-message.ts): Fix logic to handle error messages and show API key modal when api_key_missing error occurs
♻️ (use-handle-websocket-message.ts): Refactor error handling logic to improve readability and maintainability
📝 (voice-assistant.tsx): Add console log for showApiKeyModal variable to debug an issue

* api_key_missing error includes key_name

*  (use-handle-websocket-message.ts): update status message when API key is missing to provide more context
🔧 (voice-assistant.tsx): add support for saving and handling API key using new API endpoints and alert messages
🔧 (voice-assistant.tsx): update logic to check for existing API key and handle saving new API key with appropriate error handling

* 🔧 (gitignore): remove unnecessary whitespace at the end of the file
🔧 (output_audio.raw): delete output_audio.raw file as it is no longer needed

* 🐛 (use-handle-websocket-message.ts): Fix logic to show API key modal only if hasOpenAIAPIKey is false
🐛 (voice-assistant.tsx): Fix missing hasOpenAIAPIKey parameter in VoiceAssistant component to prevent runtime errors

* [autofix.ci] apply automated fixes

* modalities

* send all openai events and homogenize eleven_client name

* 🔧 (voice_mode.py): Refactor get_or_create_elevenlabs_client function to accept user_id and session parameters for better flexibility and reusability
🔧 (voice-assistant.tsx): Add local storage utility functions to handle saving and retrieving audio settings for voice assistant
🔧 (voiceStore.ts): Create a new store to manage voice-related data such as available voices and providers
🔧 (audio-settings-dialog.tsx): Introduce a new component to manage audio settings for the voice assistant, including provider and voice selection
🔧 (settings-voice-button.tsx): Implement a button component to open the audio settings modal for the voice assistant

* unnecessary barge in queue and elif fix

* queue_service, remove old ws endpoint, and vad tweaks

* traceback and better system prompt

* [autofix.ci] apply automated fixes

* merge

* build_flow fix

* fix flow_and_stream args

* mcp flow_and_stream

* build_flow_and_stream

* retry prompt and conversation_id

* [autofix.ci] apply automated fixes

* prompt

* switch to openai and elevenlabs & add token based auth

* 📝 (voice_mode.py): Add import statements for new modules and classes
📝 (voice_mode.py): Add WebSocket endpoint parameter for session_id
📝 (voice_mode.py): Add logic to save message to database in WebSocket endpoint
 (use-get-messages-mutation.ts): Add function to fetch messages data from API
♻️ (use-start-conversation.ts): Update WebSocket URL to include session_id
♻️ (use-start-recording.ts): Add function to handle fetching messages after recording
♻️ (streamProcessor.ts): Add error handling for already registered processor
♻️ (voice-assistant.tsx): Add function to trigger fetching messages after recording
📝 (new-modal.tsx): Set and update current session ID in modal
📝 (utilityStore.ts): Add currentSessionId state and setter function
📝 (index.ts): Add type definition for currentSessionId in utility store

* [autofix.ci] apply automated fixes

*  (voice_mode.py): add functionality to save user input messages to the database during a conversation in the voice mode feature. This allows for better tracking and analysis of user interactions.

* [autofix.ci] apply automated fixes

* continue to support ws without session_id

* intercept and replace response.done when elevenlabs. and better logging

* 🐛 (voice_mode.py): Fix issue where message was not being saved to the database in certain cases
🐛 (use-start-recording.ts): Fix bug where handleGetMessagesMutation was not being called in all scenarios

* close openai ws when client ws closes

* [autofix.ci] apply automated fixes

* rename elevenlabs event type

* move session to global, improve event error logging

* [autofix.ci] apply automated fixes

* session update merging

* key check fix

* fix session update forwarding to openai

* [autofix.ci] apply automated fixes

* 📝 (chat-view.tsx): Add chat history display functionality to show messages in the chat view component and handle file uploads and drag and drop functionality.

*  (inputGlobalComponent): Add GeneralDeleteConfirmationModal and GeneralGlobalVariableModal components for reusability and better code organization
📝 (audio-settings-dialog.tsx): Add userElevenLabsApiKey prop to SettingsVoiceModal component for ElevenLabs API key configuration
🔧 (delete-confirmation-modal.tsx): Create GeneralDeleteConfirmationModal component for reusability in delete confirmation modals
🔧 (global-variable-modal.tsx): Create GeneralGlobalVariableModal component for reusability in global variable modals

* 🐛 (voice_mode.py): Fix error saving message to database and add transcript extraction function
🔧 (constants.ts): Update calculation for LANGFLOW_ACCESS_TOKEN_EXPIRE_SECONDS to use environment variable with fallback value
♻️ (audio-settings-dialog.tsx): Refactor imports and remove unused code, improve logic for setting voice provider and voice selection
🔧 (vite.config.mts): Add dotenv package for loading environment variables, update configuration to use environment variables for BACKEND_URL, ACCESS_TOKEN_EXPIRE_SECONDS, CI, and ELEVENLABS_API_KEY

* [autofix.ci] apply automated fixes

* handle BrokenResourceError

* handle BrokenResourceError

* voice UI/UX improvements

*  (voice-assistant.spec.ts): add test for voice assistant feature to ensure it is visible and interactive on the page

*  (frontend): add feature flag for enabling voice assistant functionality
📝 (input-wrapper.tsx): conditionally render VoiceButton based on ENABLE_VOICE_ASSISTANT flag to control visibility of voice assistant feature

*  (button.tsx): Add new button variant 'outlineAmber' for better visual consistency
🔧 (feature-flags.ts): Enable voice assistant feature by setting ENABLE_VOICE_ASSISTANT to true
🔧 (voice-assistant.tsx): Update logic to initialize audio only if hasOpenAIAPIKey is true
🔧 (voice-assistant.tsx): Update styling for voice assistant container and recording indicator
🔧 (voice-assistant.tsx): Update button variant to 'outlineAmber' for settings icon
📝 (index.css): Add new CSS variables for accent-amber and red-foreground colors for better theming
📝 (tailwind.config.mjs): Add 'red-foreground' color variable to Tailwind CSS config for consistency

* 🐛 (feature-flags.ts): fix ENABLE_VOICE_ASSISTANT flag to be set to false instead of true

* 🔧 (voice_mode.py): add global dictionary to store queues for each session and track active message processing tasks
🔧 (voice_mode.py): add message processing queue to ensure ordered processing of messages in the database

*  (frontend): enable voice assistant feature flag
♻️ (frontend): refactor code to use functional update pattern in useBarControls hook
🐛 (frontend): fix logic in VoiceAssistant component to properly handle recording and initialization of audio assistant

* [autofix.ci] apply automated fixes

*  (audio-settings-dialog.tsx): add LanguageSelect component to allow users to select preferred language for speech recognition
📝 (audio-settings-dialog.tsx): add documentation for ALL_LANGUAGES constant and SettingsVoiceModalProps interface
🔧 (language-select.tsx): create LanguageSelect component for selecting preferred language for speech recognition
🔧 (use-start-recording.ts): add autoGainControl and sampleRate options for better audio recording quality
🔧 (use-start-recording.ts): set fftSize property on analyserRef for improved audio analysis
🔧 (use-start-recording.ts): pass preferredLanguage to input_audio_buffer.append event for language identification
🔧 (voice-assistant.tsx): add support for setting and saving preferred language for speech recognition

* UI adjustments

* Small fixes for client/LF session update merges

* [autofix.ci] apply automated fixes

* 🔧 (use-bar-controls.ts): Add useRef import to fix missing dependency in useBarControls hook
🔧 (use-bar-controls.ts): Add animationFrameRef and timeDataRef to manage animation and sound detection
🔧 (use-bar-controls.ts): Add support for sound detection using analyserRef and setSoundDetected
🔧 (use-bar-controls.ts): Update useEffect dependencies for better performance
🔧 (voice-assistant.tsx): Add soundDetected state to manage sound detection in VoiceAssistant component
🔧 (voice-assistant.tsx): Pass analyserRef and setSoundDetected to useBarControls hook in VoiceAssistant component
🔧 (chat-message.tsx): Update className logic to handle isAudioMessage condition in ChatMessage component

* 🔧 (use-bar-controls.ts): refactor useBarControls hook to improve readability and maintainability by introducing useRef for baseHeights, lastRandomizeTime, and minHeight, and optimizing the sound detection logic.

*  (use-get-messages-polling.ts): Add new file for handling messages polling functionality
♻️ (use-bar-controls.ts): Refactor useBarControls hook to make setSoundDetected optional
♻️ (use-start-recording.ts): Remove handleGetMessagesMutation function call
♻️ (voice-assistant.tsx): Refactor VoiceAssistant component to use useGetMessagesPollingMutation instead of useGetMessagesMutation and manage soundDetected state using useVoiceStore
📝 (chat-message.tsx): Remove unused import and type ChatMessageType
🔧 (voiceStore.ts): Add soundDetected state and setSoundDetected function to voiceStore
🔧 (voice.types.ts): Add soundDetected boolean type and setSoundDetected function to VoiceStoreType

* 🔧 (frontend): add Button component import to improve code organization
🔧 (frontend): add support for saving OpenAI API key in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponentType interface for flexibility
🔧 (frontend): add commandWidth prop to InputComponent in InputComponent index file
🔧 (frontend): add commandWidth prop to CustomInputPopover in popover index file
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component
🔧 (frontend): add commandWidth prop to InputComponent in Settings

* Update session instructions with more clear categories. Change how default LF session and client sessions are merged.

*  (chat-view.tsx): Add support for sound detection in chat view component to enhance user experience and interaction with the application.

* [autofix.ci] apply automated fixes

*  (audio-settings-dialog.tsx): Add debounce functionality to handleOpenAIKeyChange to improve performance and reduce API calls
🔧 (audio-worklet-processor.js): Add audio worklet processor for voice activity detection to enhance voice assistant functionality
♻️ (use-start-recording.ts): Refactor useStartRecording hook to only send non-silent audio data to the server for processing
🔧 (voice-assistant.tsx): Remove unnecessary import and setShowSettingsModal call in handleClickSaveOpenAIApiKey to clean up code

* 🐛 (audio-worklet-processor.js): Adjust noise and activation thresholds for better sensitivity and accuracy in speech detection
🐛 (audio-worklet-processor.js): Scale RMS volume to match use-bar-controls.ts scale for consistent threshold comparison
🐛 (use-bar-controls.ts): Update threshold for sound detection to align with scaled volume values for accurate detection

* 🔧 (use-start-recording.ts): refactor useStartRecording hook to simplify audio data handling and sending process to WebSocket server

* 🐛 (use-bar-controls.ts): adjust sound detection threshold to improve accuracy and responsiveness

* [autofix.ci] apply automated fixes

* 🐛 (voice-assistant.tsx): Update condition to initialize audio only if not recording, has API key, and settings modal is not shown
🐛 (chat-view.tsx): Fix syntax error in rendering new chat section in ChatView component

* ♻️ (chat-view.tsx): remove unnecessary parentheses in ChatView component to improve code readability

* 🐛 (voice-assistant.tsx): fix useEffect dependency array to include showSettingsModal to prevent unnecessary re-renders
♻️ (voice-assistant.tsx): remove duplicate onClick handler for showSettingsModal to improve code readability and maintainability

* 🐛 (voice-assistant.tsx): fix voice assistant settings button not closing audio input when clicked

* 🔧 (audio-settings-dialog.tsx): Remove unused 'open' prop from SettingsVoiceModal component
♻️ (voice-assistant.tsx): Refactor logic to handle audio recording and settings modal visibility
🔧 (voice-assistant.tsx): Replace AudioSettingsDialog component with SettingsVoiceModal component in VoiceAssistant component

* 🐛 (audio-settings-dialog.tsx): reduce debounce timeout from 2000ms to 1000ms for faster response to user input
🐛 (audio-settings-dialog.tsx): adjust alignOffset value in DropdownMenuContent component to -54 for better alignment on the UI

*  (button.tsx): add medium size variant to button component
🔧 (audio-settings-dialog.tsx): add isEditingOpenAIKey boolean prop and setIsEditingOpenAIKey function prop to SettingsVoiceModal component
🔧 (voice-assistant.tsx): add isEditingOpenAIKey state and setIsEditingOpenAIKey function to VoiceAssistant component, update handleSaveApiKey function to handle editing OpenAI API key

* 🐛 (audio-settings-dialog.tsx): Fix issue where the "Save" button text was not updating correctly based on editing state
📝 (voice-assistant.tsx): Add missing semicolon to the code to prevent syntax error

* 🐛 (audio-settings-dialog.tsx): Fix issue with setIsEditingOpenAIKey function call
♻️ (use-bar-controls.ts): Refactor code to properly initialize and handle audio analyzer
 (voice-assistant.tsx): Introduce logic to properly initialize audio and start recording when voice assistant is activated

* 🐛 (audio-settings-dialog.tsx): fix typo in setIsEditingOpenAIKey function call to properly close the editing of OpenAI key when modal is closed

* 🔧 (audio-settings-dialog.tsx): change onClick handler function name from setOpen to onOpenChangeDropdownMenu for clarity and consistency

* lint fixes

* ruff fixes

* [autofix.ci] apply automated fixes

*  (test_voice_mode.py): import pathlib module for improved file path handling
♻️ (test_voice_mode.py): refactor code to use newer numpy random Generator for better random number generation
📝 (test_voice_mode.py): update comments for clarity and consistency in test functions

* 🔧 (components.py): refactor global variables to use a class for managing component cache
🔧 (components.py): replace os module with pathlib for file path operations
🔧 (components.py): refactor global variables to use a class for managing fully loaded components
🔧 (components.py): refactor function to write audio bytes to file using a helper function

* 🔧 Refactor voice_utils.py: Improve error handling in write_audio_to_file function

*  (voice-select.tsx): add unique key prop to SelectItem component to avoid React warning and improve performance

* [autofix.ci] apply automated fixes

*  (audio-settings-dialog.tsx): Update onClick function to setIsEditingOpenAIKey instead of onOpenChangeDropdownMenu for better clarity
📝 (microphone-select.tsx): Update text from "Input" to "Audio Input" for better user understanding
♻️ (voice-assistant.tsx): Refactor handleSaveApiKey function to handle both OpenAI and ElevenLabs API keys, improving code readability and maintainability. Remove unnecessary checks for already saved API keys.

* 📝 (voice_utils.py): add docstring to resample_24k_to_16k function for clarity and documentation purposes
📝 (voice_utils.py): add input validation to ensure frame_24k_bytes is exactly 960 bytes before resampling
📝 (test_voice_mode.py): refactor test_webrtcvad_with_real_data to generate synthetic audio data for testing instead of reading from a file
📝 (test_voice_mode.py): update comments and assertions for clarity and accuracy in speech detection testing

* [autofix.ci] apply automated fixes

* 🐛 (feature-flags.ts): fix duplicate declaration of ENABLE_VOICE_ASSISTANT variable

* cache config by session_id

* [autofix.ci] apply automated fixes

*  (audio-settings-dialog.tsx): Add conditional rendering for MicrophoneSelect component based on the presence of microphones array to prevent rendering when no microphones are available
🐛 (microphone-select.tsx): Add optional chaining to navigator.mediaDevices calls to prevent errors when navigator or mediaDevices are null or undefined
🐛 (use-start-recording.ts): Add optional chaining to navigator.mediaDevices calls to prevent errors when navigator or mediaDevices are null or undefined

* ♻️ (voice_mode.py): refactor openai_realtime_session attribute to explicitly define its type as a dictionary with string keys and any values for better code clarity and type safety

* 📝 (voice_mode.py): add Optional import from typing module to improve type hinting
♻️ (voice_mode.py): make ElevenLabs package optional to avoid errors if not installed and provide a fallback mechanism

* [autofix.ci] apply automated fixes

* ♻️ (voice_mode.py): refactor ElevenLabs and ApiError classes to include type ignore comments to suppress redefinition warnings

* [autofix.ci] apply automated fixes

* merge

* event logger to debug

* 🐛 (language-select.tsx): Fix potential null reference error when accessing lang object properties
🐛 (voice-select.tsx): Fix potential null reference error when accessing voice object properties
🐛 (stringManipulation.ts): Fix potential null reference error when calling toLowerCase and toUpperCase functions

* fix openai voices

* error handling

* frontend socket state

* [autofix.ci] apply automated fixes

* add session to event logging

* add elevenlabs to base package

* 🐛 (audio-settings-dialog.tsx): Fix rendering issue with MicrophoneSelect component
🐛 (microphone-select.tsx): Fix potential null pointer exception in rendering microphone label
🐛 (voice-assistant.tsx): Fix issue with toggling recording functionality to properly handle microphone stream and tracks

* 🐛 (voice-assistant.tsx): fix optional chaining for microphoneRef to prevent potential null pointer errors

* [autofix.ci] apply automated fixes

* 🐛 (use-start-conversation.ts): fix variable name from sessionId to currentSessionId for clarity and consistency
🔧 (use-start-conversation.ts): refactor WebSocket connection URL to use current host, port, and protocol for better flexibility and compatibility

* alternating messages

* mypy

* [autofix.ci] apply automated fixes

*  (userSettings.spec.ts): add additional wait times to improve stability and reliability of tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
Co-authored-by: nfreybler <nfreybler@nvidia.com>
2025-03-20 00:05:55 +00:00

36 lines
673 B
Plaintext

# Set the default behavior, in case people don't have core.autocrlf set.
* text eol=lf
# Explicitly declare text files you want to always be normalized and converted
# to native line endings on checkout.
*.c text
*.h text
*.py text
*.js text
*.jsx text
*.ts text
*.tsx text
*.md text
*.mdx text
*.yml text
*.yaml text
*.xml text
*.csv text
*.json text
*.sh text
*.Dockerfile text
Dockerfile text
# Declare files that will always have CRLF line endings on checkout.
*.sln text eol=crlf
# Denote all files that are truly binary and should not be modified.
*.png binary
*.jpg binary
*.ico binary
*.gif binary
*.mp4 binary
*.svg binary
*.csv binary
*.wav binary
*.raw binary