Shehab Ashraf

Contributed to Open Superintelligence Lab's 5-Dollar LLM, an open-source project to train LLM on a minimal compute budget.

Open Source Contributor at Open Superintelligence Lab

Contributed to Open Superintelligence Lab's 5-Dollar LLM, an open-source project to train LLM on a minimal compute budget.

• PR #67 (Merged): Computed loss inside the forward pass to avoid returning full logit tensors and enabled Fused AdamW, boosting training throughput by 13% (14.7k to 16.6k tokens/sec) on an NVIDIA L4.
• PR #88 (Merged): Merged Q, K, V, and O projection layers into a single contiguous parameter tensor, boosting training throughput by 32% (80.5k to 106.6k tokens/sec) on an NVIDIA A100.

Bachelor of Science in Computer and Systems Engineering – Zagazig University, Faculty of Engineering (2018 – 2023)