• Developed for NTU MLDA Deep Learning Week Hackathon in 48 hours (1/10/2022 - 3/10/2022).
• Achieved 1st place out of 120 teams.
• Detailed explanation of our solution: https://me.tjh.sg/blog/bytevid

Project Description

Say goodbye to long and boring videos! 👋

Powered by the cutting-edge deep learning technologies in 2022, ByteVid transforms long, boring videos into fun byte-sized content.

Be it a one hour long lecture, or a 30-minute zoom meeting, ByteVid can transcribe, summarise the content, extract keywords, detect and extract important slides from the video, and translate into other languages.

Deep Learning

• Whisper: SOTA speech recognition (Sep 2022)
• YOLOv7: SOTA object detection (Jul 2022)
• KBIR-inspec: key phrase extraction (Dec 2021)
• Bert Extractive Summarizer: summarisation (Jun 2019)
• BlingFire: sentence extraction
• Baidu Translate API: translation

Frontend

• React.js
• Tailwind CSS
• Deploy on GitHub pages

Backend

• Flask server
• Deploy on a GPU machine
• Relay to an Internet-facing VPS
• Nginx reverse proxy
• Cloudflare protection

Tools

• OpenCV
• youtube-dl
• ffmpeg

• Developed a mobile-responsive web app that visualises the attention mechanism of Transformer-based model (BERT) in HTML, CSS, JavaScript, and D3.js
• Executed the BERT model directly in browser (client-side)
• Received more than 28 stars on the GitHub repository

• Developed a Python library that implements a modified, lightweight version of HuggingFace BERT Tokenizer in pure Python
• My tokenizer can maintain high performance in resource-limited devices like embedded systems and web browsers
• My tokenizer is 57% faster than the original HuggingFace BERT Tokenizer