[AINews] Microsoft AgentInstruct + Orca 3 • ButtondownTwitterTwitter

This section provides a recap of AI-related discussions on Twitter. It covers a shooting incident at a Trump rally, reactions and commentary from various individuals, new AI and ML research developments including top papers of the week and recent AI implementations and applications by different organizations.

New APIs and services

• @virattt launched an open beta stock market API with 30+ years of data for S&P 500 tickers, including financial statements, with no API limits. It's undergoing load testing before a full 15,000+ stock launch for AI financial agents to utilize.

• Coding experiences and tips: @giffmana shared frustration with unhelpful online resources when writing a Python script to read multipart/form-data, finding the actual RFC2388 spec most useful. @jeremyphoward demonstrated a new function-cache decorator design in Python to compose cache eviction policies.

• Developer discussions: @svpino predicted AI becoming a foundational skill for future developers alongside data structures and algorithms, as software development and machine learning converge.

OpenAI is developing a new reasoning technology named Strawberry, drawing comparisons to Stanford's STaR.

JavaScript Juggles: LangChain's Trio of Functions:

Users dissected the intricacies of LangChain JS's invoke, stream, and streamEvents, debating their efficacy for streaming outputs in langgraph.

• A proposal emerged suggesting the use of agents for assorted tasks like data collection and API interactions.

Base64 Blues with Gemini API: Seek, Decode, Fail:

A puzzling 'invalid input' snag was hit when a user wielded Base64 with Gemini Pro API, despite File API uploads being the lone documented method.

• The collective's guidance pointed towards the need for clarity in docs and further elaboration on Base64 usage with APIs.

ToolCall Toss-up: LangChain’s Legacy to OpenAIToolCall:

ToolCall, now obsolete, directs users to its successor OpenAIToolCall, introducing an index feature for order.

• The community pondered package updates and the handling of auto mode's inadvertent default tool calls.

Hallucination Hazards: Chatbots Conjure Queries:

Hallucinations in HuggingFace models were reported, provoking discussions around the LLM-generated random question/answer pairs for chatbots.

• Alternative remedies were offered, including a shift to either openAI-models or FireworksAI models, although finetuned llama models seemed resilient to the typical repetition penalties.

Embedding Excellence: OpenAI Models Spotlight:

Curiosity peaked over the optimal OpenAI embedding model, sparking a discourse on the best model to comprehend and utilize embedding vectors.

• The general consensus leaned towards text-embedding-ada-002 recommended as the go-to model in LangChain for vector embeddings.

• A member is interested in training a hybrid model using EfficientNetB7 for feature extraction and Swin transformer for classification, using Google Colab due to limited computational power.
• Another member is facing installation issues of OpenPose on an Ubuntu laptop without GPU and without CUDA, encountering a CMake error.

• A new notebook was shared to help users easily finetune LLMs using CSV data, broadening the scope of data manipulation and finetuning tasks.
• Members shared strategies for managing training checkpoints effectively, emphasizing the importance of consistent training setups to avoid issues with seed shuffling impacting resume-from-checkpoint functionality.

The section discusses various topics related to LM Studio and hardware interactions. It includes discussions on issues like hardware performance with AI models, multi-GPU systems, Mac versus custom PCs for AI, ROCm and OpenCL support, and PCIe bandwidth implications. Members share insights on potential improvements using Intel Arc a750, the importance of ROCm support for AMD GPUs, choosing GPUs for LM Studio, navigating multi-GPU setups for AI, and considerations when selecting hardware for local AI setups. The section also includes mentions of various links related to hardware discussions.

The discussions in this section shed light on various aspects related to CoPilots and AI tools. Users debated the comparison between Copilot and Bing’s AI, shared experiences on Word and PowerPoint CoPilots' functionality, addressed challenges with offline model execution, and provided tips for customizing and training AI models. Additionally, there were discussions on exploring affordable AI tools like GPT-3.5 and the alternatives to GPT-4. The section also included links to further resources on topics like Python RAG Tutorial and the OpenAI Developer Forum.

Members of the Nous Research AI channel engaged in various discussions related to AI models, updates, and industry trends. Some topics included improving Large Language Models (LLMs) reasoning, speculation on OpenAI's new platform updates, and exploring Google's new Vision Language Model. Additionally, Teknium announced a hiring search, and discussions on integrating LLMs into apps for tutorials and extending context length for models. Overall, the conversations showcased a diverse range of interests and inquiries within the AI community.

The Nous Research AI section discusses various topics related to the RAG dataset, including Marker version speedup, integration with synthetic RAG, XML in agent definition, Mixture of Agents models, and Stasima diverse models. Marker's new version is significantly faster on different platforms due to efficient architecture, designed for converting PDFs to Markdown. XML was mentioned to simplify defining agents. A member showcased a Mixture-of-Agents implementation in just 50 lines of code, integrating multiple models. Another member discussed the concept in their project Stasima, using different prompts to create a spectrum of agents.

This section discusses the exploration of new tools and technologies related to WebGPU, Transformers.js, and troubleshooting Dawn build issues. Members shared resources for learning WebGPU and the use of Transformers.js for browser-based machine learning tasks. Troubleshooting strategies for Dawn build on Windows were discussed, along with limitations of WebGPU buffers in browsers. Experiences with WebGPU setup and suggestions for improvements were also shared.

• Memorable Acronym: 3E: A suggestion for using a more memorable acronym like Extract, Evaluate, Extend/Expand (3E) was made.
• Demand for More Demos and Examples: The need for more demos and examples in technical discussions was emphasized by multiple members.
• Exploring Evaluation Techniques: Members discussed evaluation techniques like logprob, GPTscore, and hyperparameter optimization tools like prompt-hyperopt. A paper titled Simple approach for contextual hallucinations was mentioned in relation to this.
• State Management Tools Comparison: A comparison of state management styles, focusing on ReAct framework, Langgraph, and XState, was discussed.
• Upcoming AI in Action Talks: Next week, VikParuchuri will present on converting PDF to Markdown using tools like marker and surya.

RAG Model Dataset Scraping Concerns:

• User nafnlaus00 raised security concerns about using Chromium to render sites requiring JavaScript, such as Quora, for creating a RAG model dataset.
• Le_mess suggested troubleshooting headers/params issues and considering services like firecrawl or the Jina API for safer scraping.

Proposing Weighted Training Data:

• Tostino suggested implementing a system for weighting different parts of conversation data in both pretraining and SFT, allowing negative weights to teach models to avoid certain tokens.
• This could enable optimization loops where less understood sections or 'bad paths' are weighted differently to improve model outcomes.

PR for Chat Template Dataset Soon:

• User announced the upcoming PR for a new chat template dataset type offering flexibility on training sections.
• This includes selecting roles to train on, configuring train_on_eos, and handling specific training sections within the dataset.

Concerns Over Stuck PR Reviews:

• A member raised concerns about PR reviews being stuck, mentioning specific PRs from themselves and another user.
• 'Are PR reviews getting stuck?' user asked, pointing to their PR and another one.

Interconnects (Nathan Lambert) ▷ #reads (4 messages):

• California AI Bill SB 1047 sparks fierce debate: The California AI Bill SB 1047, passed in the state's Senate, is heading to a final vote amidst intense lobbying. State senator Scott Wiener described the debate as 'Jets vs Sharks'.

• Paywall circumvention using Archive.is: A method to bypass paywalls using Archive.is was discussed, allowing access to content behind paywalls like those on Fortune.

• Links Mentioned:

  • It's AI's 'Sharks vs. Jets'—welcome to the fight over California's AI safety bill
  • California AI bill SB-1047 sparks fierce debate around regulation of …

• A member requested an API key for OpenAI for a chatbot project, mentioning the need for creating a tutorial.
• Another member sought unused OpenAI API keys specifically for tutorial purposes.