AIFINEX
  • Giriş
  • Başlarken
  • AIFINEX Flow'u Kullanma
    • Agentflows
      • Multi-Agents
      • Sequential Agents
    • API
    • Analitik
    • Belge Depoları
    • Telemetry
  • Yapılandırma
    • Doğrulama
      • Sohbet Akışı Seviyesi
    • Oran Sınırı
  • Integrations
    • LangChain
      • Agents
        • Airtable Agent
        • AutoGPT
        • BabyAGI
        • CSV Agents
        • Conversational Agent
        • OpenAI Assistant
          • Threads
        • ReAct Agent Chat
        • ReAct Agent LLM
        • Tool Agent
        • XML Agent
      • Cache
        • InMemory Cache
        • InMemory Embedding Cache
        • Momento Cache
        • Redis Cache
        • Redis Embeddings Cache
        • Upstash Redis Cache
      • Chains
        • GET API Chain
        • OpenAPI Chain
        • POST API Chain
        • Conversation Chain
        • Conversational Retrieval QA Chain
        • LLM Chain
        • Multi Prompt Chain
        • Multi Retrieval QA Chain
        • Retrieval QA Chain
        • Sql Database Chain
        • Vectara QA Chain
        • VectorDB QA Chain
      • Chat Models
        • AWS ChatBedrock
        • Azure ChatOpenAI
        • NIBittensorChat
        • ChatAnthropic
        • ChatCohere
        • Chat Fireworks
        • ChatGoogleGenerativeAI
        • ChatGooglePaLM
        • Google VertexAI
        • ChatHuggingFace
        • ChatLocalAI
        • ChatMistralAI
        • ChatOllama
        • ChatOllama Function
        • ChatOpenAI
        • ChatTogetherAI
        • GroqChat
      • Document Loaders
        • API Loader
        • Airtable
        • Apify Website Content Crawler
        • Cheerio Web Scraper
        • Confluence
        • CSV File
        • Custom Document Loader
        • Document Store
        • Docx File
        • Figma
        • FireCrawl
        • Folder with Files
        • GitBook
        • Github
        • Json File
        • Json Lines File
        • Notion Database
        • Notion Folder
        • Notion Page
        • PDF Files
        • Plain Text
        • Playwright Web Scraper
        • Puppeteer Web Scraper
        • S3 File Loader
        • SearchApi For Web Search
        • SerpApi For Web Search
        • Spider - web search & crawler
        • Text File
        • Unstructured File Loader
        • Unstructured Folder Loader
        • VectorStore To Document
      • Embeddings
        • AWS Bedrock Embeddings
        • Azure OpenAI Eklemeleri
        • Cohere Embeddings
        • Google GenerativeAI Embeddings
        • Google PaLM Embeddings
        • Google VertexAI Embeddings
        • HuggingFace Inference Embeddings
        • LocalAI Embeddings
        • MistralAI Embeddings
        • Ollama Embeddings
        • OpenAI Embeddings
        • OpenAI Embeddings Custom
        • TogetherAI Embedding
        • VoyageAI Embeddings
      • LLMs
        • AWS Bedrock
        • Azure OpenAI
        • NIBittensorLLM
        • Cohere
        • GooglePaLM
        • GoogleVertex AI
        • HuggingFace Inference
        • Ollama
        • OpenAI
        • Replicate
      • Memory
        • Buffer Memory
        • Buffer Window Memory
        • Conversation Summary Memory
        • Conversation Summary Buffer Memory
        • DynamoDB Chat Memory
        • MongoDB Atlas Chat Memory
        • Redis-Backed Chat Memory
        • Upstash Redis-Backed Chat Memory
        • Zep Memory
      • Moderation
        • OpenAI Moderasyonu
        • Basit İstem Moderasyonu
      • Output Parsers
        • CSV Çıktı Ayrıştırıcısı
        • Özel Liste Çıktı Ayrıştırıcısı
        • Yapılandırılmış Çıktı Ayrıştırıcısı
        • Gelişmiş Yapılandırılmış Çıktı Ayrıştırıcısı
      • Prompts
        • Chat Prompt Template
        • Birkaç Atış İstemi Şablonu
        • İstem Şablonu
      • Record Managers
      • Retriever
        • Cohere Rerank Retriever
        • Embeddings Filter Retriever
        • HyDE Retriever
        • LLM Filter Retriever
        • Multi Query Retriever
        • Prompt Retriever
        • Reciprocal Rank Fusion Retriever
        • Similarity Score Threshold Retriever
        • Vector Store Retriever
        • Voyage AI Rerank Retriever
      • Text Splitters
        • Character Text Splitter
        • Code Text Splitter
        • Html-To-Markdown Text Splitter
        • Markdown Text Splitter
        • Recursive Character Text Splitter
        • Token Text Splitter
      • Tools
        • BraveSearch API
        • Calculator
        • Chain Tool
        • Chatflow Tool
        • Custom Tool
        • Exa Search
        • Google Custom Search
        • OpenAPI Toolkit
        • Python Interpreter
        • Read File
        • Request Get
        • Request Post
        • Retriever Tool
        • SearchApi
        • SearXNG
        • Serp API
        • Serper
        • Web Browser
        • Write File
      • Vector Stores
        • AstraDB
        • Chroma
        • Elastic
        • Faiss
        • In-Memory Vector Store
        • Milvus
        • MongoDB Atlas
        • OpenSearch
        • Pinecone
        • Postgres
        • Qdrant
        • Redis
        • SingleStore
        • Supabase
        • Upstash Vector
        • Vectara
        • Weaviate
        • Zep Collection - Open Source
        • Zep Collection - Cloud
    • LiteLLM Proxy
    • LlamaIndex
      • Agents
        • OpenAI Tool Agent
        • Anthropic Tool Agent
      • Chat Models
        • AzureChatOpenAI
        • ChatAnthropic
        • ChatMistral
        • ChatOllama
        • ChatOpenAI
        • ChatTogetherAI
        • ChatGroq
      • Embeddings
        • Azure OpenAI Embeddings
        • OpenAI Embeddings
      • Engine
        • Query Engine
        • Simple Chat Engine
        • Context Chat Engine
        • Sub-Question Query Engine
      • Response Synthesizer
        • Refine
        • Compact and Refine
        • Simple Response Builder
        • Tree Summarize
      • Tools
        • Query Engine Tool
      • Vector Stores
        • Pinecone
        • SimpleStore
    • Utilities
      • Custom JS Function
      • Set/Get Variable
      • If-Else
      • Sticky Note
    • External Integrations
      • Zapier Zaps
  • Use Cases
    • Calling Children Flows
    • Calling Webhook
    • Interacting with API
    • Multiple Documents QnA
    • SQL QnA
    • Upserting Data
    • Web Scrape QnA
Powered by GitBook
On this page
  1. Integrations
  2. LangChain
  3. Text Splitters

Token Text Splitter

Ham bir metin dizesini önce metni BPE (Byte Pair Encoding) belirteçlerine dönüştürerek böler, ardından bu belirteçleri parçalara ayırır ve tek bir yığın içindeki belirteçleri tekrar metne dönüştürür.

PreviousRecursive Character Text SplitterNextTools

Last updated 8 months ago

Bu bölüm devam eden bir çalışmadır. Bu bölümü tamamlarken sağlayabileceğiniz her türlü yardım için teşekkür ederiz.

Token Text Splitter Düğümü