8️⃣LlmaPack

LlmaPack 사용법

%pip install pypdf -q -U

What is LlamaPack?

Llama Packs은 사전 패키지된 모듈/템플릿의 커뮤니티 중심 허브

Llama Hubllamahub.ai

LlamaPack 사용 2가지 방법

1. CLI

llamaindex-cli download-llamapack <pack_name> --download-dir <pack_directory>

2. Python

from llama_index.core.llama_pack import download_llama_pack

pack_cls = download_llama_pack("<pack_name>", "<pack_directory>")

Use Cases

RAG 파이프라인 개발에서 유용한 LlmaPack 4가지를 소개합니다.

Document 준비

• 데모 목적으로 Optimism과 StarkNet의 분석 및 비교 PDF 문서를 사용하겠습니다.

• Analysis and Comparison between Optimism and StarkNet

!wget -O Analysis_and_Comparison_between_Optimism_and_StarkNet.pdf https://ceur-ws.org/Vol-3460/papers/DLT_2023_paper_16.pdf

from llama_index.readers.file import PDFReader

reader = PDFReader()
documents = reader.load_data(
    "Analysis_and_Comparison_between_Optimism_and_StarkNet.pdf"
)

len(documents)

14

from llama_index.core.node_parser import SimpleNodeParser

nodes = SimpleNodeParser.from_defaults().get_nodes_from_documents(documents)

len(nodes)

14

import os
from dotenv import load_dotenv  

!echo "OPENAI_API_KEY=<Your OpenAI Key>" >> .env # OpenAI API Key 입력
load_dotenv()
api_key = os.getenv("OPENAI_API_KEY")

from llama_index.core.llama_pack import download_llama_pack

import nest_asyncio

nest_asyncio.apply()

QUESTION = "유효성 롤업의 목표는 무엇이야?"

HybridFusionRetrieverPack

• Vector retriever

• BM25 retriever

llama_pack = download_llama_pack(
    "HybridFusionRetrieverPack", "./hybrid_fusion_pack"
)

hybrid_fusion_pack = llama_pack(
    nodes,
    chunk_size=256,
    vector_similarity_top_k=2,
    bm25_similarity_top_k=2
)

response = hybrid_fusion_pack.run(QUESTION)

Generated queries:
1. 유효성 롤업의 장단점은 무엇인가?
2. 유효성 롤업이 왜 중요한가?
3. 유효성 롤업을 효과적으로 구현하는 방법은?

response

Response(response='The goal of Validity Rollups is to ensure the finalization of withdrawals on Layer 1 as soon as the validity proof of the new state root is accepted on Ethereum.', source_nodes=[NodeWithScore(node=TextNode(id_='960bba74-4039-4cce-b62b-a89764a6bced', embedding=None, metadata={'page_label': '3', 'file_name': 'Analysis_and_Comparison_between_Optimism_and_StarkNet.pdf'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='85a01680-d7be-475d-9355-51aa0504602b', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '3', 'file_name': 'Analysis_and_Comparison_between_Optimism_and_StarkNet.pdf'}, hash='162138014c4d86025fcd3966a59c200affab1200ec9a8f16c8e545d32b05cf51')

response.response

'The goal of Validity Rollups is to ensure the finalization of withdrawals on Layer 1 as soon as the validity proof of the new state root is accepted on Ethereum.'

QueryRewritingRetrieverPack

이 팩은 사용자의 쿼리를 기반으로 여러 쿼리를 생성합니다.

llama_pack = download_llama_pack(
    "QueryRewritingRetrieverPack", "./query_rewriting_pack"
)

query_rewriting_pack = llama_pack(
    nodes,
    chunk_size=256,
    vector_similarity_top_k=2,
)

response = query_rewriting_pack.run(QUESTION)

Generated queries:
1. 유효성 롤업의 중요성은 무엇인가?
2. 유효성 롤업의 장단점은 무엇인가?
3. 유효성 롤업을 효과적으로 구현하는 방법은 무엇인가?

response.response

'The goal of Validity Rollups is to enable the finalization of withdrawals on Layer 1 as soon as the validity proof of the new state root is accepted on Ethereum, allowing for the quick withdrawal of funds.'

SentenceWindowRetrieverPack

SentenceWindowRetrieverPack은 문서를 로드하고, 청크로 묶고, 각 청크에 메타데이터로 주변 컨텍스트를 추가하고, 검색 중에 응답 합성을 위해 각 청크에 컨텍스트를 다시 삽입합니다.

#%pip install 'pymemgpt[local]'==0.3.6
#%pip install --upgrade llama-index-embeddings-huggingface

llama_pack = download_llama_pack(
    "SentenceWindowRetrieverPack", "./sentence_window_retriever_pack"
)

# Pass documents to build the pack instance
sentence_window_retriever_pack = llama_pack(documents)

response = sentence_window_retriever_pack.run(QUESTION)

response.response

'The goal of Validity Rollups is to enable the finalization of withdrawals on Layer 1 as soon as the validity proof of the new state root is accepted on Ethereum, allowing for the quick withdrawal of funds.'

response

Response(response='The goal of Validity Rollups is to enable the finalization of withdrawals on Layer 1 as soon as the validity proof of the new state root is accepted on Ethereum, allowing for the quick withdrawal of funds.', source_nodes=[NodeWithScore(node=TextNode(id_='960bba74-4039-4cce-b62b-a89764a6bced', embedding=None, metadata={'page_label': '3', 'file_name': 'Analysis_and_Comparison_between_Optimism_and_StarkNet.pdf'},

RecursiveRetrieverSmallToBigPack

RecursiveRetrieverSmallToBigPack은 문서를 로드하고 계층적 노드 그래프(더 큰 부모 노드와 더 작은 자식 노드)를 작성합니다.

llama_pack = download_llama_pack(
    "RecursiveRetrieverSmallToBigPack", "./recursive_retriever_stb_pack"
)

recursive_retriever_stb_pack = llama_pack(documents)

response = recursive_retriever_stb_pack.run(QUESTION)

response.response

'The goal of Validity Rollups is to enable the finalization of withdrawals on Layer 1 as soon as the validity proof of the new state root is accepted on Ethereum, allowing for the quick withdrawal of funds.'

LLamaPack 모듈의 개별적 사용

Packe에 포함된 모듈을 가져와서 개별적으로 사용할 수 있습니다.

# get the sentence vector index
index = sentence_window_retriever_pack.sentence_index

# get the node parser
node_parser = sentence_window_retriever_pack.node_parser

# get the metadata replacement postprocessor
postprocessor = sentence_window_retriever_pack.postprocessor

# get the query engine
query_engine = sentence_window_retriever_pack.query_engine

index

node_parser

postprocessor

query_engine