A guide to ingesting custom data like VM tags, script outputs, and application logs into Azure Log Analytics for enhanced monitoring and alerting.
An introduction to data engineering concepts, focusing on data sources and ingestion strategies like batch vs. streaming.
Explores methods for ingesting Debezium CDC events from Kafka into Apache Flink using different SQL connectors and data formats.
A technical guide exploring different Flink SQL connectors and formats for ingesting and processing Debezium CDC events from Apache Kafka topics.
A guide to setting up and using Dremio's Auto-Ingest feature for automated, event-driven data loading into Apache Iceberg tables from cloud storage.
A guide to building a real-time data pipeline using Azure services like Event Hub, Stream Analytics, Parquet, ADLS, and ADX.
A technical guide on setting up Azure Event Hub to ingest and route compressed data into Azure Data Explorer (ADX) for real-time analytics.
A guide to installing and using LightIngest on Ubuntu for data ingestion into Azure Data Explorer, including managed identity authentication.
Explains how to use Azure OpenAI's 'Add your data' feature programmatically via REST APIs to integrate custom documents into AI responses.
A guide to building a custom chatbot in Azure OpenAI using your own data, covering setup, costs, and integration with Azure Cognitive Search.
A technical guide on building a real-time air quality monitoring and alert system using Azure cloud services and PurpleAir sensor data.
A technical guide on using kafkacat to load static CSV data into Apache Kafka topics for data enrichment, focusing on simplicity over Kafka Connect.
Day 6 of a series on Kafka Connect Single Message Transforms, focusing on using InsertField to add static values and Kafka metadata to messages.
A technical guide on using Kafka Connect with Single Message Transform to ingest and process XML data into Apache Kafka.
A technical guide on using Unix pipelines with curl, xq, and kafkacat to ingest and transform XML data into JSON for Kafka.
Introduction to ingesting XML data into Kafka, covering challenges and approaches for handling XML schemas in Kafka topics.
A technical guide on using Kafka Connect's SpoolDir connector to load CSV data into Apache Kafka, covering schema application and configuration.
A tutorial on building a Telegram bot using Apache Kafka and ksqlDB to process, query, and respond to real-time data streams.
Explores methods for copying data into Azure Synapse Analytics (SQL DW), focusing on the CTAS and new COPY INTO commands.
A tutorial on monitoring Logstash data ingest rates using InfluxDB for storage and Grafana for visualization.
