[AINews] Gemini Live • ButtondownTwitterTwitter

This section provides a detailed recap of discussions and developments related to AI on Reddit. It covers topics such as advanced quantization and model optimization techniques, open-source contributions to LLM development, and the release of new AI models. Highlights include the successful quantization of the Llama-3.1 70B model to 4-bit HQQ/calibrated quantized model, the efficiency of the Unsloth model in handling tasks on limited GPU memory, and the pre-training of a 1.3B parameter language model in just 9 days using Retro-GPT. Additionally, the section discusses the open-source repository with different RAG implementations and the release of the Falcon Mamba 7B SSLM by the Technology Innovation Institute. Overall, these discussions showcase a mix of technical achievements, community contributions, and evaluations of new AI models on Reddit.

AI Model Releases and Capabilities

• Speculation about new GPT-4 model: A post on r/singularity claims ChatGPT mentioned a new GPT-4o model out since last week, generating discussion about potential new OpenAI releases.
• Flux image generation model: Several posts discuss the capabilities of the new Flux image generation models, including impressive impressionist landscape generation, attempts at generating anatomically correct nude images, and creative ad concept generation for fictional products.

AI-Generated Media

• AI-generated video with synthetic voice: A demo video shows Flux-generated images animated and paired with AI-generated voice, though commenters note issues with lip sync and voice quality.

Autonomous Vehicles

• Waymo self-driving car issues: A video post shows Waymo autonomous vehicles having difficulties navigating from their starting point, sparking discussion on current limitations.

AI and Society

• AI companions and relationships: A controversial meme post sparked debate about the potential impact of AI companions on human relationships and societal dynamics.

This section covers various discussions from different Discord channels related to the optimization processes of the 'Java by Microsoft' project, updates in Coher For AI Research Lab, Torchtune Discord discussions on TransformerDecoderLayer refactor and DPO and PPO testing, OpenAccess AI Collective's discussions on Perplexity Pro and Llama 3 MoE, and LAION Discord updates on Grok 2.0, Flux.1 model, and 2D pooling success. Additionally, it includes MLOps at Chipro Discord talks on Poe's partnership with Agihouse for a hackathon, and LangChain AI conversation on Llama 3.1 8b's structured output capabilities. Moreover, insights from the OpenInterpreter Discord on Open Interpreter in Obsidian and AI agents in enterprises, and discussions from Alignment Lab AI Discord on SlimOrca models and fine-tuning effects are presented. Lastly, the LLM Finetuning Discord sections focus on automation systems for Jupyter Notebooks, and a 'help' section covers issues with Unsloth model loading/saving, finetuning Llama 3.1 model in Hindi, using Unsloth with VLLM, creating custom datasets, and memory usage problems.

A member proposed an alternative approach to faster model initialization using curand to initialize weights directly on the GPU. This PR is still under development and requires further testing and clean-up.

Vision Adapters: The Key to Vision Models

Only specific LLM models have vision adapters, most of them are going by name 'LLaVa' or 'obsidian'.

• The 'VISION ADAPTER' is a crucial component for vision models; without it, the error you shared will pop up.

Mistral Large: The Current Champion?

A member found Mistral Large 2 to be the best LLM right now, outcompeting Claude 3.5 Sonnet for difficult novel problems.

• However, the member also noted that Gemini Flash undercut OpenAI 4o mini severely in price, but OpenAI 4o was less expensive than Mistral Large.

LLM Studio's Model Explorer is Down

Several members reported that HuggingFace, which powers the LM Studio Model Explorer, is down.

• The site was confirmed to be inaccessible for several hours, with connectivity issues reported across various locations.

Llama 3.1 Performance Issues

A user reported that their Llama 3 8B model is now running at only 3 tok/s, compared to 15 tok/s before a recent update.

• The user checked their GPU offload settings and reset them to default, but the problem persists; the issue appears to be related to a change in the recent update.

LLM Output Length Control

A member is looking for ways to restrict the output length of responses, as some models tend to output whole paragraphs even when instructed to provide a single sentence.

• While system prompts can be modified, the member found that 8B models, specifically Meta-Llama-3.1-8B-Instruct-GGUFI, are not the best at following precise instructions.

OpenRouter (Alex Atallah) announces updates for Gemini Flash 1.5 and GPT-4o Extended. Gemini Flash token costs have decreased by 78% making it more accessible. GPT-4o Extended early access has launched. The maximum token limit for GPT-4o Extended is 64k. Users compare the performance of Groq 70b and Hyperbolic. ChatGPT 4.0 default setting changes prompt user concerns.### Mojo discussions revolve around licensing, open-sourcing, development focus, and learning resources. Users discuss memory issues with Stable Diffusion in WSL2 and recommend increasing memory allocation.### Cohere discussions highlight the launch of Cohere For AI research lab, pricing changes inquiries, readiness for hackathons, and interests in computer vision.### Torchtune developers discuss TransformerDecoderLayer refactor, RLHF with DPO/PPO, performance issues with DPO config, and PyTorch Conference updates.### OpenAccess AI Collective Axolotl users engage in discussions about Perplexity Pro's reasoning abilities, Llama 3 adaptations, grad clipping explanation, and OpenAI benchmark release strategy.

Open Interpreter for Anything to Anything:

Use Open Interpreter to convert any type of data into any other format.

• This is possible by using the "Convert Anything" tool, which harnesses the power of Open Interpreter.

Open Interpreter in Obsidian:

A new YouTube series is launching that will demonstrate how to use Open Interpreter in the Obsidian note-taking app.

• This plugin allows you to control your Obsidian vault using Open Interpreter, which could have major implications for how people work with knowledge.

The footer section of the website includes links to find AI News on various social networks and platforms. Users can connect through Twitter or subscribe to the newsletter through a provided link. The footer also mentions that the website is brought to you by Buttondown, offering an easy way to start and grow newsletters.