AI, ML systems, operating systems, and software engineering.
I'm Karthik Gangula, a Year 1 NTU Data Science and AI student and also part of the Turing AI Scholars Programme at NTU . I'm interested in ML systems and AI research, and I'm always eager to learn, grow, and look for opportunities to contribute in these fields.
This is my technical blog, where I write about AI, machine learning systems, operating systems, and other software engineering topics.
Benchmarked and profiled LLM inference on single and multi-node (A100, H100) systems using vLLM, SGLang, and Tensorrt (w/ triton inference server) leveraging MLPerf Inference (Datacenter Suite) for offline, online, and interactive scenarios.
Identified and resolved performance bottlenecks via Nsight Systems, Nsight Compute, and PyTorch Profiler; optimized throughput and latency through quantization of weights, activations, and KV cache.
Contributed improvements and findings to MLPerf, vLLM, and SGLang GitHub repositories; authored a technical report guiding optimization strategies for production-scale LLM inference.
Doing research and development to predict hepatocellular carcinoma (HCC) development using advanced deep learning models on histology images, focusing on “time to event” analysis.
Worked primarily with Python, OpenCV and TensorFlow to preprocess and build deep learning models for survival analysis using large scale image datasets.
If you want to get in touch with me about something or just to say hi, reach out on social media.
Karthik Gangula