General AI
Algorithm: A set of instructions that a computer follows to perform a specific task.
Artificial Intelligence (AI): This is the overarching term for algorithms that can exhibit intelligent behavior, like learning, reasoning, and problem-solving.
Artificial General Intelligence (AGI): A theoretical, advanced form of AI designed to understand, learn, and apply knowledge across any intellectual task at a level equal to or exceeding human capabilities.
Natural Language (NL): The language humans use to communicate with each other, including spoken and written forms.
Natural Language Processing (NLP): A field of AI that focuses on algorithms being able to interpret, manipulate, and comprehend human language.
Computer Vision: This is a field of AI concerned with algorithms that understand and process visual information from the real world.
Large Language Model (LLM): A powerful AI model trained on massive amounts of text data. LLMs can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.
Chatbots: A computer program that simulates a conversation between human users and a Large Language Model.
Multimodal: A description of a model that allows users to integrate multiple input layers, including text, images, and video, to generate more comprehensive and contextually rich outputs. This multidimensional approach enables AI systems to understand and interpret complex scenarios with greater depth and sophistication.
Temperature: A parameter that controls the randomness of the generated text. A higher temperature leads to more creative and surprising outputs, but also potentially less factual or relevant. A lower temperature results in more conservative and predictable responses that stay closer to the established context.
Tokens: The numerical representation of the input data that an AI model processes. For text, prompts are broken down into words and subwords and then given number IDs that match with those elements. Tokens also represent other input types such as image, video, audio, etc.
Context Window: This defines the amount of information an AI model can consider when generating a response. This is measured by token count.
Hallucination: When an AI generates inaccurate, biased, or otherwise unintended information as a response.
Benchmark Evaluation: Standardized tests, datasets, and metrics used to objectively measure and compare the performance, accuracy, and capabilities of AI models or systems. These benchmarks help researchers and developers compare different models and assess their strengths and weaknesses.
Open Source: Refers to software or data that is freely available for anyone to use, modify, and distribute. Open source usually refers to making every part of development available. Most local AI models are described as open weight instead of open source because their training datasets and model architecture isn’t always disclosed.
Fine-tuning: This is a technique used to improve the performance of a large language model on a specific task. It involves taking a pre-trained model and further training it on a smaller dataset focused on the specific task you want it to perform better at.
Prompt Engineering
Prompt Engineering: This is the art of crafting instructions and context for an AI model to get the desired response. It’s like carefully phrasing a question to get a specific answer from a human.
Prompt Template: This is a pre-defined structure or format for guiding the LLM in generating a response. It often includes placeholders for user input, instructions for the model, and formatting specifications for the output.
Zero Shot Prompting: An AI technique where a LLM performs a task without being given any task-specific examples. This makes the model rely on its pre-trained knowledge to understand instructions and generate output.
Few Shot Prompting: This technique supplies example prompt and response pairs for the AI to understand the task better and follow the output format.
Chain of Thought Prompting: This technique breaks down complex tasks into a series of steps (all in one prompt), allowing the AI to reason through the problem with all of the required steps.
Least to Most Prompting: This technique is a strategy where you start with a very basic prompt and gradually add more details or instructions until you get the desired response from the AI model.
Prompt Chaining: A technique that breaks down a prompt into multiple subtasks. Prompt chaining is typically used for longer generations or complex tasks that need more direction in its execution.
Iterative Prompting: This technique refines your prompt for an AI model in stages, based on the initial response you receive. You essentially have a conversation with the model, providing feedback and additional information to guide it towards the desired output.
System Prompt: A foundational, behind-the-scenes meta-prompt that defines an AI model’s persona, behavior, rules, and constraints before any user interaction occurs.
Tool Calling / Agents
Function/Tool Calling: The ability of a model to trigger pre-defined functions or tools within the system. These functions can access external data sources or perform specific tasks to enhance the model’s response.
Retrieval Augmented Generation (RAG): A method of connecting a large language model with an external datasource. This is usually focused on private or unorganized data and the utilization of a vector database.
Indexing: Formatting a database so specific data can be retrieved based on a query (structured or natural language).
Embeddings: A vector (or list) of floating point numbers acting as numerical representations of data points. They capture the semantic relationships between data points, allowing for faster retrieval and comparison based on meaning rather than just keywords.
Vector Database: This is a database specifically designed to store and efficiently search for vector embeddings. These databases are optimized for similarity searches, allowing you to find data points with similar meaning to a query embedding.
K Nearest Neighbor Search (KNN): A machine learning algorithm that finds the k closest data points (neighbors) in a dataset to a new, unclassified data point. The distance metric (e.g., cosine similarity) determines how “close” neighbors are.
Agents: An AI model that chooses the sequence of actions to take. This allows the model to take control and complete more versatile tasks instead of being hard coded down a path in a chain.
Application Programming Interface (API): An API acts as a bridge between software applications. It provides a set of defined functions and instructions that allow different programs to communicate and exchange data.
API Key: This is a unique identifier used to authenticate access to an API. It acts like a key or a password that unlocks the functionalities offered by the API.
Model Context Protocol (MCP): An open standard that enables developers to build secure, two-way connections between their data sources and AI Agents.
Computational Processing
Central Processing Unit (CPU): The most common processors. They are like the brain of a computer, handling a wide range of tasks in a sequential manner. CPUs feature a small number of powerful cores optimized for executing complex instructions with low latency.
Graphics Processing Unit (GPU): A specialized processor designed for high-performance graphics calculations. GPUs are well-suited for parallel processing tasks and excel at accelerating deep learning training due to their large number of cores optimized for matrix operations.
Tensor Processing Unit (TPU): A specialized hardware accelerator developed by Google specifically for accelerating machine learning workloads. TPUs are optimized for tensor operations, which are fundamental to many deep learning models. They offer higher throughput and energy efficiency compared to traditional GPUs for certain AI tasks.
Quantum Processing Unit (QPU): The cutting edge of computing technology, leveraging the principles of quantum mechanics to perform computations at exponentially faster rates than classical computers. While still in the early stages of development, QPUs hold immense promise for solving complex AI problems with unprecedented speed.
Local Model: A machine learning model trained and run entirely on a user’s device, processing data locally without needing a remote server. This offers privacy benefits and offline functionality but may be limited by device storage and processing power.
Compute Unified Device Architecture (CUDA): CUDA is a prominent parallel computing platform and programming model developed by NVIDIA. It empowers developers to tap into the vast computational power of NVIDIA GPUs for a wide range of tasks, including AI applications like deep learning. CUDA offers a suite of programming tools and libraries that enable developers to effectively parallelize computations and offload them to the GPU.
Cloud Computing Platforms: A service that provides on-demand access to computing resources like servers, storage, databases, networking, and tools over the internet, without the need for extensive physical infrastructure. Examples include Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform.
Edge Devices: Computing devices located at the periphery of a network, closer to where data is generated. Processing data on edge devices can reduce latency and improve efficiency for AI applications that require real-time decision-making.
Quantization: Reducing the numerical precision of model weights and activations (typically from 32-bit floating-point to 8-bit integers) to decrease model size, speed up inference, and lower memory usage.
Machine Learning
Training: The process of teaching an AI model to gain intelligence by feeding it data, analyzing the results, and adjusting the model’s output to improve its accuracy and efficiency.
Supervised Learning: Training that relies on labeled data. Labeled data is the paired input and output data that is used during training to calculate the cost, which is optimized by adjusting the weights and biases.
Unsupervised Learning: Training that deals with unlabeled data. The data points don’t have predefined labels or categories. The goal is to uncover hidden patterns or structures within the data itself.
Reinforcement Learning (RL): This is a type of machine learning where an agent learns through trial and error in an interactive environment. The agent receives rewards for taking actions that move it closer to a desired goal and penalties for actions that don’t. Over time, the agent learns to choose actions that maximize its long-term reward.
Reinforcement Learning from Human Feedback (RLHF): This builds upon regular reinforcement learning by incorporating human feedback into the training process. Instead of solely relying on pre-defined rewards, RLHF allows humans to guide the agent’s learning by providing feedback on its actions.
Neuron: The processing units of the network. Each receives input from other neurons, performs a calculation, and transmits a single output value.
Weight: A numerical value associated with the connection between two neurons. Weights determine the strength of the signal transmitted between neurons and influence how much one neuron’s output affects another.
Bias (Neural Network Element): A constant value added to the weighted sum of inputs within a neuron. The bias allows the neuron to shift its activation function and adjust its output independently of the weighted input.
Parameter Size: This refers to the total number of adjustable variables within an AI model. These parameters are what the model learns and adjusts during training to improve its performance.
Forward Pass: Forward propagation (or forward pass) refers to the calculation and storage of intermediate variables (including outputs) for a neural network in order from the input layer to the output layer.
Cost/Loss Function: A function that quantifies how “wrong” the network’s prediction is after the forward pass. The cost function is the average error of n-samples. The loss function is the error of a single sample. Cost and loss are often used interchangeably.
Metric: A quantitative measure used to evaluate the performance of a machine learning model on a specific task. Common metrics include accuracy, precision, recall, and F1 score for classification tasks, and mean squared error (MSE) or R-squared for regression tasks.
Backpropagation: Using a technique called backpropagation, the network adjusts its weights and biases in a backward fashion based on the cost. The adjustments aim to minimize the overall cost, essentially “teaching” the network how to improve its predictions for future examples.
Gradient Descent: An optimization algorithm commonly used in conjunction with backpropagation. It iteratively adjusts the weights and biases of a neural network in the direction that minimizes the cost function.
Hyperparameter: These are settings that control the learning process of a machine learning model. They are not directly learned from the data but are set by the developer beforehand.
Batch Size: The number of training examples processed together in a single forward pass. Choosing an appropriate batch size can impact training speed and stability. Common choices include batch sizes of 1, 32, 64, or 128.
Epoch: One complete pass through the entire training dataset. During training, a model might go through multiple epochs, iteratively refining its weights and biases based on the entire set of training examples.
Overfitting: A situation where a machine learning model memorizes the training data too well and fails to generalize to unseen data. This results in poor performance on new data points not encountered during training.
Models
Model Architecture: The overall structure and design of a machine learning model, including the type of neural network used, the number of layers and neurons, and the activation functions employed.
Feed Forward Neural Network (FNN): The simplest model, which serves as the foundation for many other architectures. They consist of stacked layers of fully connected neurons, where information flows in a single, forward direction from input to output.
Convolutional Neural Network (CNN): A specialized type of neural network particularly effective for image recognition and analysis. CNNs utilize filters (kernels) that convolve with the input data, extracting features like edges and shapes.
Recurrent Neural Network (RNN): A specialized type of neural network designed to handle sequential data (where order matters) like text, speech, or time series data. The basic RNN unit consists of a single recurrent neuron with connections that loop back on themselves, allowing information to persist across the network, enabling them to “remember” what they’ve processed previously.
Long Short-Term Memory (LSTM): A specialized type of neural network equipped with a more sophisticated memory cell compared to traditional RNNs. They are capable of learning long-range dependencies in sequential data. They have mechanisms called memory gates to selectively remember or forget information over time.
Exploding Gradient Problem: The recurrent weight is exponentially tied to the number of sequential inputs there are. So a weight with a bigger value can skyrocket depending on the amount of sequential inputs.
Vanishing Gradient Problem: The recurrent weight is exponentially tied to the number of sequential inputs there are. So a weight with a value below one can diminish to almost zero.
Encoder: Like a compressor. It takes a piece of information (like an image, text, or a sequence of data) and converts it into a more compact representation. This compressed representation, often called a code or a latent space, captures the essential features of the input data in a smaller format.
Decoder: Like an expander. It takes the compressed code generated by the encoder and attempts to reconstruct the original data or generate something similar to it.
Autoencoder: An encoder network and a decoder network that reconstructs the input from the latent representation. Autoencoders learn to compress data into a lower-dimensional representation and then regenerate the original data from that compressed version (known as a bottleneck layer).
Denoising Autoencoder (DAE): A type of autoencoder neural network specifically designed to remove noise from data. It learns to reconstruct a clean version of the input data by filtering out the noise during the encoding and decoding process.
Variational Autoencoder (VAE): A type of autoencoder that learns a compressed representation (latent space) of data. VAEs are useful for tasks like dimensionality reduction, anomaly detection, and generating new data samples similar to the training data.
Transformer: This model revolutionized natural language processing tasks by relying solely on attention mechanisms. This allows the model to focus on crucial parts of the input sequence, critical for grasping context and relationships between data. They rely on an encoder-decoder structure.
Attention: Attention mechanisms enable neural networks to focus on specific parts of the input data while processing sequences, allowing for more effective information processing
Self-Attention: Self-Attention focuses solely on the relationships within a single input sequence. The model essentially “attends” to different parts of its own input to understand the context and relationships between them.
Multi-Head Attention: An extension of self-attention that allows the model to learn different attention patterns from the input data simultaneously. This enhances the model’s ability to capture diverse and complementary aspects of the information.
Diffusion Model: A model that creates realistic data (images, audio, text) by learning to reverse a process of gradually adding noise, essentially turning random static back into structured content, used in tools like DALL-E and Stable Diffusion for high-quality image generation from text prompts.
Platforms
ChatGPT: An OpenAI chatbot application that utilizes their large language models.
Google AI Studio: A web-based application powered by Google’s Gemini models that gives more advanced access to generation parameters such as system prompts and temperature.
n8n: A powerful, node-based workflow automation tool that allows users to connect web applications, databases, and AI models.
Ollama: An open-source tool that simplifies running, managing, and interacting with large language models directly on your local computer for privacy, speed, and offline use.
Open WebUI: A popular, self-hosted, user-friendly AI platform that provides a ChatGPT-like interface for interacting with various large language models (LLMs), supporting local models.
GitHub: A web-based developer platform that uses the Git version control system to store, manage, and share code. It is the world’s largest source code host and provides a collaborative environment for individuals and teams to work on software projects.
Hugging Face: The GitHub for AI. It holds repositories of open-source models, datasets, and tools that cater to a diverse range of processing needs.
LangChain: This is an open-source library specifically designed for building AI assistants. It provides tools and functionalities for managing prompts, responses, integrating different models and tools, and facilitating communication between the assistant and the user.
Teachable Machine: A free, web-based tool from Google that lets anyone build machine learning models for images, sounds, and poses without coding, making AI accessible for creating custom classifiers for websites, apps, and projects by training them with your own data directly in the browser.
Ethics and Safety
AI Ethics: A branch of ethics concerned with the responsible development, deployment, and use of artificial intelligence. It considers issues like fairness, transparency, accountability, privacy, and safety in AI systems.
Bias (Ethical): A disproportionate weight in favor of or against an idea or thing, usually in a way that is inaccurate, closed-minded, prejudicial, or unfair.
Explicit Bias: Conscious attitudes, beliefs, or prejudices that individuals are aware of and can articulate. These biases are often deliberate and intentional, reflecting conscious beliefs or stereotypes about certain groups of people.
Implicit Bias: Also known as unconscious bias, refers to attitudes, beliefs, or stereotypes that influence our judgments and behaviors at a subconscious level, often without our awareness. These biases operate automatically and involuntarily, shaping our perceptions and decision-making processes.
Selection Bias: This happens when the data used to train an AI system is not representative of the reality it’s meant to model. It can occur due to various reasons, such as incomplete data, biased sampling, or other factors that may lead to an unrepresentative dataset.
Confirmation Bias: This type of bias happens when an AI system is tuned to rely too much on pre-existing beliefs or trends in the data. This can reinforce existing biases and fail to identify new patterns or trends.
Stereotyping Bias: Generalizing about entire groups of people, leading to inaccurate predictions for individuals within those groups. AI systems can perpetuate stereotypes if trained on biased data.
Measurement Bias: Bias introduced by the way data is measured or interpreted. For example, using flawed metrics to evaluate the performance of an AI system can lead to biased conclusions.
Shadow AI: Unauthorized AI usage in the workplace, presents significant security risks and compliance challenges. From data breaches to algorithmic biases, the unauthorized deployment of AI technologies can have serious consequences.
Adversarial Attacks: Malicious actors can exploit vulnerabilities in AI systems through manipulation techniques to trick AI algorithms into making incorrect or harmful decisions, leading to security breaches or data manipulation.
Malware: Malicious software designed to harm a computer system or steal data. Malware can include viruses, worms, ransomware, and spyware.
Misinformation: False or inaccurate information that is spread, regardless of whether there is intent to deceive. Misinformation can be harmful as it can mislead people and influence their decisions.
Deepfake: A type of synthetic media that uses deep learning techniques to create realistic but manipulated videos or audio recordings. Deepfakes can be used for entertainment purposes but also pose a threat for spreading misinformation.
Lethal Autonomous Weapons System (LAWS): AI technology can be integrated into autonomous weapons systems, enabling the development of lethal drones, robots, or cyber weapons capable of making life-and-death decisions without human intervention, raising ethical and humanitarian concerns.
Algorithmic Trading Manipulation: AI algorithms can be exploited to manipulate financial markets, conduct high-frequency trading, or perpetrate pump-and-dump schemes, leading to market manipulation, instability, and financial fraud.