Activity

1. RobinQu

   added this to First GA Release of instinct.cppon Apr 3, 2024
2. RobinQu

   converted this from a draft issue on Apr 11, 2024
3. RobinQu

   moved this to In Progress in First GA Release of instinct.cppon Apr 11, 2024
4. RobinQu commented on Apr 11, 2024

   

   RobinQu

   on Apr 11, 2024

   Last edited by RobinQu

   OwnerAuthor

   More actions

   Agent Protocol

   The protocol

   https://agentprotocol.ai/protocol

   Restful Endpoints:

   • Tasks: Create,List,Retrieve
   • Step: Run, Get, List
   • Artifact: Upload,Download

   Demo Code

   server code:

   from agent_protocol import Agent, Step, Task
   
   
   async def plan(step: Step) -> Step:
       task = await Agent.db.get_task(step.task_id)
       steps = generate_steps(task.input)
   
       last_step = steps[-1]
       for step in steps[:-1]:
          await Agent.db.create_step(task_id=task.task_id, name=step, ...)
   
       await Agent.db.create_step(task_id=task.task_id, name=last_step, is_last=True)
       step.output = steps
       return step
   
   
   async def execute(step: Step) -> Step:
       # Use tools, websearch, etc.
       ...
   
   async def task_handler(task: Task) -> None:
       await Agent.db.create_step(task_id=task.task_id, name="plan", ...)
   
   
   async def step_handler(step: Step) -> Step:
       if step.name == "plan":
           await plan(step)
       else:
           await execute(step)
   
      return step
   
   
   Agent.setup_agent(task_handler, step_handler).start()
   

   client code:

   import asyncio
   
   from agent_protocol.models import StepRequestBody
   from agent_protocol_client import (
       Configuration,
       ApiClient,
       StepRequestBody,
       TaskRequestBody,
       AgentApi,
   )
   
   # Defining the host is optional and defaults to http://localhost
   # See configuration.py for a list of all supported configuration parameters.
   configuration = Configuration(host="http://localhost:8000")
   
   
   async def main():
       # Enter a context with an instance of the API client
       async with ApiClient(configuration) as api_client:
           # Create an instance of the API class
           api_instance = AgentApi(api_client)
           task_request_body = TaskRequestBody(input="Write 'Hello world!' to hi.txt.")
   
           response = await api_instance.create_agent_task(
               task_request_body=task_request_body
           )
           print("The response of AgentApi->create_agent_task:\n")
           print(response)
           print("\n\n")
   
           task_id = response.task_id
           i = 1
   
           while (
               step := await api_instance.execute_agent_task_step(
                   task_id=task_id, step_request_body=StepRequestBody(input=i)
               )
           ) and step.is_last is False:
               print("The response of AgentApi->execute_agent_task_step:\n")
               print(step)
               print("\n\n")
               i += 1
   
           print("Agent finished its work!")
   
           artifacts = await api_instance.list_agent_task_artifacts(task_id=task_id)
           for artifact in artifacts:
               if artifact.file_name == "hi.txt":
                   content = await api_instance.download_agent_task_artifact(
                       task_id=task_id, artifact_id=artifact.artifact_id
                   )
                   print(f'The content of the hi.txt is {content})')
                   break
           else:
               print("The agent did not create the hi.txt file.")
   
   asyncio.run(main())

   Adoptions

   https://github.com/ai-engineer-foundation/agent-protocol?tab=readme-ov-file#open-source-agents-and-projects-that-have-adopted-agent-protocol

   
5. RobinQu commented on Apr 11, 2024

   

   RobinQu

   on Apr 11, 2024

   Last edited by RobinQu

   OwnerAuthor

   More actions

   OpenAI Assistant API

   The protocol

   https://platform.openai.com/docs/api-reference/assistants

   Endpoints

   • Assistants
     • abstraction of model config, file resources, tools.
     • actions: Create, List, Delete, Update
   • Threads
     • ongoing task information: messages, files, metadata
     • actions: Create, Retrieve, update
   • Messages
     • messages in threads
     • Action: Create, List,
   • Runs
     • Actions: Create, List, ListSteps, Get, Submit tool output, Cancel

   Resource objects

   

   Lifecycle of run objects

   https://platform.openai.com/docs/assistants/how-it-works/run-lifecycle

   

   

   • Expriation: 10mins after function call is requested.
   • function call result is saved to run_step.step_details

   Tools

   Assistant API defines three primary tools.

   • code interpreter: solve problem to run Python code in sandbox
   • retriever: it will ingest uploaded files with assistant object, or thread object.
   • function calling: external tool usage.

   They are general purpose tools. I suppose more specific tools will come, e.g. data interpreter.

   Adoptions

   • https://lablab.ai/apps/tech/openai/assistants-api
   • https://github.com/davideuler/awesome-assistant-api?tab=readme-ov-file

   Open sourced implementations

   https://github.com/transitive-bullshit/OpenOpenAI
   https://github.com/stellar-amenities/hal-9100/tree/main

   Related readings

   • https://medium.com/@2twitme/crafting-intelligent-user-experiences-a-deep-dive-into-openai-assistants-api-00439ace108a
   • https://github.com/supershaneski/openai-assistants-api-multi-user-sample
6. RobinQu commented on Apr 11, 2024

   

   RobinQu

   on Apr 11, 2024

   Last edited by RobinQu

   OwnerAuthor

   More actions

   Conclusion

   API Design	Pros	Cons
   Agent Protocol	* API endpoints are concise and clear. * Have official test suites and benchmarks.	* Low-level: it has no concept of tools, memory, etc. Some extra work needed to implement a high-level protocol above it. * limited current adoptions
   OpenAI Assistant API	* Rich abstraction of agent execution flow and related tools. * GPT-4 models are really good at general instructions for agent. * Relatively richer features in client sdk. * built-in tools: code interpreter, retriever, and function calling.	* Closed sourced. we don’t know how the the underlying agent architecture is built. And little information about API server is revealed on documentation websites. * a little bit complicated API design, esp. for Run, RunStep stuff. Pretty sure that these APIs will be improved in future.

   Bottom lines:

   • OpenAI’s solution is easy to adapt and more powerful in terms of foundation models and built-in tools.
   • Influence of OpenAI results more downstream projects based on Assistant API, which may be the next software ecosystem (GUI, benchmarks, etc. ) like its ChatCompletionAPI.
   • Agent protocol is more friendly for agent developers, not for agent users, at least for now. It’s definitely a potential opponent to OpenAI’s solution.

   Regarding instinct.cpp:

   • My bet is on Assistant API.
   • More analysis of implementation cost is needed before further development.
7. RobinQu commented on Apr 19, 2024

   

   RobinQu

   on Apr 19, 2024

   Last edited by RobinQu

   OwnerAuthor

   More actions

   OpenAI's Assistant V2 is released: https://platform.openai.com/docs/assistants/migration

   • API updates on assistant, thread, message.
   • retriever becomes filesearch, which support more sophiscated control over vector stores.
8. RobinQu commented on Apr 28, 2024

   

   RobinQu

   on Apr 28, 2024

   Last edited by RobinQu

   OwnerAuthor

   More actions

   tool call test with assitant api

   test_function_call_sync_1714287406.json

   Observations:

   • No thought text is generated. function call options are genearted directly. Could they ommit thought text and use action text directly? Or they have different prompt techniques for tool uses.
   • in event stream, thread.run.step.completed is duplicated.
   • run is queued again after tool outputs are submitted.

   API with function calls:

   • https://platform.openai.com/docs/guides/function-calling
   • mixtral api: https://medium.com/@thomas_82964/function-calling-with-mistral-7b-ce81a8715c93

   Tool agent using OpenAI style tool call API:

   • https://github.com/run-llama/llama_index/blob/40913847ba47d435b40b7fac3ae83eba89b56bb9/llama-index-core/llama_index/core/agent/runner/parallel.py#L352
   • https://github.com/langchain-ai/langgraph/blob/main/langgraph/prebuilt/chat_agent_executor.py

   Conclusion

   • These tool agent relies on API with function call.
   • When last message does not contain function call request, then the conversion is regarded as finallized. This is simple but probematic for many reasons.
   • Other stop strategy:
     • if max step number are reached, stop abrutly with no final answer.
     • if a user provided special function tool is called, agent will accpet call result as final answer and then stop.
9. RobinQu

   moved this from In Progress to Done in First GA Release of instinct.cppon May 20, 2024
10. RobinQu commented on May 20, 2024

    

    RobinQu

    on May 20, 2024

    OwnerAuthor

    More actions

    it should be closed long ago with v01.2.0 release.
11. RobinQu

    closed this as completedon May 20, 2024