20 | The Future of AI and Robotics Begins Now, Grok is Open, RIP to Chatbot PI.

Our Menu :)

• NVIDIA GTC: The Future of AI and Robotics Starts Now.

• Grok is Open Source: Elon fulfilled its promise

• Inflection is eaten alive by its biggest investor. 

• Exec Summary of 2hrs chat of Sam Altman with Alex Fridman 

Actionable Tips:

• Essentials for better prompting: RACEF Framework 

Read time: 6 Minutes

GEN AI STARTUP
NVIDIA GTC: The Future of AI and Robotics Starts Now

Mar 18th, 2024. NVIDIA kicked off the GPU Technology Conference (GTC), and its CEO Jensen Huang unveiled a lot during this keynote.  

What’s new?

NVIDIA's game-changing H100 AI chip catapulted it into a multitrillion-dollar league, possibly outpacing giants like Alphabet and Amazon. And now, with the introduction of the new Blackwell B200 GPU and the GB200 "superchip," NVIDIA seems poised to widen its lead even further, leaving competitors in the dust as it races ahead.

Blackwell B200 GPU

NVIDIA has revealed the Blackwell B200 GPU, touted as the 'world's most powerful chip' for AI. This cutting-edge chip is designed to broaden access to trillion-parameter AI models, further solidifying NVIDIA's dominance in the AI market.

Key Features:

• Performance: The B200 boasts an impressive 20 petaflops of FP4 performance, thanks to its 208 billion transistors.

• Efficiency: When combined with a Grace CPU as a GB200, it provides 30x the LLM inference workload performance, leading to a cost and energy reduction of up to 25x compared to the H100.

• Capacity: It can train a massive 1.8 trillion parameter model using just 2,000 GPUs and four megawatts, a significant improvement over the previous requirement of 8,000 Hopper GPUs.

• Speed: The B200 demonstrates a 7x performance increase on GPT-3 LLM (175 billion parameters) and trains 4x faster than the H100.

Project GR00T: General Robotics zero zero Three 

NVIDIA also introduced Project GR00T, which focuses on creating foundational models for humanoid robots. This initiative enables robots to comprehend and act on multimodal instructions, such as language, video, and demonstrations.

Highlights of Project GR00T:

• Foundation Model: Empowers robots to understand and process multimodal instructions.

• Cross-Platform Compatibility: Functions seamlessly across various robotic platforms.

• Tech Stack: Leverages Isaac Lab for simulations, scales models with OSMO, and utilizes Jetson Thor for edge computing tasks.

• GEAR Lab Integration: Strives to develop adaptable agents that perform effectively in both virtual and real-world environments.

NIM Microservices:

Streamlining AI Deployment Additionally, NVIDIA launched NIM microservices to simplify the deployment of AI. These microservices enable rapid development and scaling of AI applications across multiple platforms.

NVIDIA's Leap into Digital Twins and Advanced Robotics

NVIDIA has created a Matrix-like world called the Omniverse to train those robots. it allows individuals and teams to create physically accurate virtual worlds for industrial and scientific use cases

The introduction of Omniverse Cloud APIs extends these capabilities, promising a transformative impact across various sectors, including automotive, robotics, and beyond.

• Companies like WPP, Media.Monks, Continental, Ansys, Cadence, Dassault Systèmes, Hexagon, Microsoft, Rockwell Automation, Siemens, and Trimble are embracing Omniverse Cloud APIs in their software portfolios.

• Siemens is integrating Omniverse Cloud APIs into its Siemens Xcelerator Platform to enhance cloud-based product lifecycle management software like Teamcenter X with generative AI capabilities for immersive digital twin experiences

• NVIDIA Omniverse has integrated with Apple Vision Pro to enable advanced 3D development and streaming capabilities.

  This collaboration allows developers to transfer their Universal Scene Description (USD) scenes to NVIDIA's Graphics Delivery Network (GDN), facilitating the streaming of immersive 3D content to Apple Vision Pro's high-resolution displays globally

GEN AI STARTUP
Grok is Open Source: Elon fulfilled its promise

 Mar 17th, 2024, xAI shook the tech industry by making Grok-1, a colossal language model with 314 billion parameters, open-source.

This Mixture-of-Experts model was not fine-tuned for any specific tasks.

One of Grok-1's most impressive features is its efficient weight usage, with only 25% of weights active for each token, boosting both efficiency and performance.

Elon Musk stayed true to his promise by open-sourcing Grok-1 following a legal battle with OpenAI over its proprietary approach, a decision that sparked controversy and accusations of hypocrisy.

Let's dive into some key specifications:

• Parameters: 314 billion, with only 25% of weights active per token.

• Architecture: A Mixture of 8 Experts, utilizing 2 per token.

• Layers: 64 transformer layers, effortlessly combining multihead attention and dense blocks.

• Tokenization: Employs a SentencePiece tokenizer, with a vocabulary size of 131,072.

• Embedding and Positional Encoding: 6,144 embedding size, paired with corresponding rotary positional embeddings.

• Context Length: Capable of handling 8,192 tokens with bf16 precision.

In terms of performance, Grok-1 outperforms its competitors:

• It beats LLaMa 2 70B and Mixtral 8x7B with an impressive MMLU score of 73%, showcasing its efficiency and accuracy across various tests.

Here are some implementation details to consider:

• Due to its massive size, Grok-1 demands significant GPU resources.

• It utilizes an inefficient MoE layer implementation to avoid the need for custom kernels, prioritizing model correctness validation.

• The model supports activation sharding and 8-bit quantization to optimize performance.

  

In a nutshell, Grok-1 is a game-changer for tech businesses, offering unparalleled performance and efficiency. With its open-source availability, it's an opportunity for tech founders to leverage this cutting-edge technology for their businesses.

GEN AI STARTUP
Inflection is eaten alive by its biggest investor.

Microsoft has made a significant move by bringing on board key members from the startup Inflection AI, including its co-founder Mustafa Suleyman, who's also known for co-founding Google's DeepMind. These new hires will be part of Microsoft's freshly established consumer AI division, Microsoft AI.

Highlights:

• Mustafa Suleyman is stepping in as the CEO of Microsoft AI, tasked with spearheading consumer AI projects including Copilot, Bing, and Edge, under the direct guidance of Microsoft's CEO, Satya Nadella.

• Joining Suleyman at Microsoft is Karen Simonyan, a close associate and long-term collaborator, who will serve as the Chief Scientist of Microsoft AI.

• Last year, Inflection AI secured $1.3B in funding (with Microsoft leading the round) for the development of the Pi chatbot.

  Despite the initial buzz, the startup faced challenges with its business model, prompting Suleyman and most of the Inflection AI team to transition to Microsoft.

Why It Matters:

Microsoft's strategic acquisition of top talent from Inflection AI, including a pivotal figure like Mustafa Suleyman, marks a bold step in its quest to dominate the consumer AI landscape.

This maneuver not only intensifies the talent wars within the AI industry but also leaves Inflection AI in a precarious position, with co-founder Reid Hoffman and the new CEO, Sean White, left to navigate the aftermath.

GEN AI STARTUP
Exec Summary of 2hrs chat of Sam Altman with Alex Fridman on its podcast.

OpenAI CEO Sam Altman interviewed by Lex Fridman discussing AI, GPT-5, Sora, Elon lawsuit, and more

On rumors of artificial general intelligence: “Ilya has not seen AGI. None of us have seen AGI. We’ve not built AGI.”

On the lawsuit from Elon Musk: “I think this whole thing is unbecoming of a builder…it makes me extra sad he’s doing [this] to us.”

On copyright and AI: “Do people who create valuable data deserve to have some way that they get compensated for use of it? I think the answer is yes. I don’t know yet what the answer is.”

On job loss as a result of AI: “The way I think about it is not what percent of jobs AI will do, but what percent of tasks will AI do on over one time horizon…maybe people are way more efficient at the job they do”

On GPT-5: “I expect that the delta between [GPT-5] and [GPT-4] will be the same as between [GPT-4] and [GPT-3]…We will release an amazing new model this year. I don’t know what we’ll call it.”

GEN AI AT WORK
Essentials for better prompting: RACEF Framework

Breaking Down RACEF

• Role: Specify the perspective or role you want ChatGPT to adopt. This sets the stage for the type of information you're seeking.

• Action: Define what you want ChatGPT to do. This clarity ensures the response is aligned with your expectations.

• Context: Give background information to contextualize your request. This helps in tailoring the response more closely to your needs.

• Example: Providing an example helps illustrate the kind of answer you're looking for, guiding the AI in the right direction.

• Format: Specify how you want the information presented. This could be a list, a detailed explanation, or any other format that suits your requirement.

RACEF in Practice

Step 1: Role

• Prompt Component: "You are a high school coding club advisor."

• Purpose: This specifies the role ChatGPT should assume, providing a clear standpoint from which to offer suggestions.

Step 2: Action

• Prompt Component: "Generate a list of project ideas."

• Purpose: This defines the specific action ChatGPT needs to perform, ensuring the response is focused on idea generation.

Step 3: Context

• Prompt Component: "The club members have intermediate programming skills and have worked with Python and JavaScript."

• Purpose: Providing this context helps tailor the response to the skill level and experience of the club members, ensuring the project ideas are suitable.

Step 4: Example

• Prompt Component: "Previously, they've enjoyed projects like building a simple web app and a basic game."

• Purpose: Including an example of past projects gives ChatGPT a reference point for the complexity and type of projects that have been successful and engaging for the club.

Step 5: Format

• Prompt Component: "List three ideas with a brief description for each."

• Purpose: Specifying the format ensures that the response is organized e.g. as Table, Json, XML, email, post, List

Full RACEF Prompt Example:

"As a high school coding club advisor, generate a list of project ideas suitable for club members who have intermediate programming skills and have previously worked with Python and JavaScript. They've enjoyed projects like building a simple web app and a basic game before. Please list three ideas with a brief description for each."

"As a content creator focusing on sustainability, I'm looking for a unique angle for my next blog post. Considering recent trends in renewable energy, can you suggest a topic that highlights innovative home energy solutions? Ideally, present it as a question to engage readers."

Thank you, see you next week!