Organizing DeerHacks | Anthony Tedja

DeerHacks III Organizers. Read the LinkedIn post

A few months ago, our team at UTM MCSS organized and hosted DeerHacks' third iteration at University of Toronto with Major League Hacking!

With the help of Shiva Mulwani, we led the development of the DeerHacks dashboard that served 1000+ applicants with 50,000+ website visitors! Here's how we did it.

Background

Here's the DeerHacks lore leading up to DeerHacks III.

DeerHacks 2022

The DeerHacks hackathon started ~~or was rebooted according to MCSS alumni~~ in 2022. I wasn't on the MCSS team yet and joined the hackathon as a participant rather than an organizer. With a couple of my friends, we won the 1st place prize. We were still in "Zoom" university because of COVID so we created a desktop client that automates Zoom actions with machine learning.

Left to right: Kevin, Hani, & I (Raghav missing lol)

After the hackathon, I wanted to join the team that brought the event to life so I applied to join the MCSS team for the following year as an associate on the tech team.

DeerHacks 2023

As a tech associate on the MCSS team, I designed and developed the DeerHacks II website with a nighttime / nature theme. The website is a static page with minimal JavaScript built with Bootstrap Studio. Everything on the website is hardcoded from the schedule modals to the stories.

print('spain'[1:])print('spain'[1:])

The tech team used Typeform and Google Forms to handle hacker registration logistics and used Discord and our website to keep hackers up to date on event details. We also used Mailchimp to handle emails and Google Sheets to store our forms data.

DeerHacks II Organizers

The organizing team worked hard to get our first MLH partnership to put DeerHacks on the map 😤. DeerHacks II had exponentially grown in scale, boasting 500+ applicants and 25 events throughout the hackathon weekend. ~~It would have been 26 if it didn't rain on Saturday and made us cancel the touching grass workshop.~~

DeerHacks has been the MCSS team's big, final event of the year. Many of the organizer's graduations were set in place, leaving many executive roles open. One of these roles included the Director of Technology, in which Shiva and I took on the role as co-directors, with Shiva leading backend and cloud services and myself leading frontend and product development.

The DeerHacks Dashboard

The DeerHacks dashboard services are open source at https://github.com/utmmcss.

The Maanjiwe Nendamowinan building model being built in blender

Ever since DeerHacks II wrapped up, our team has been planning on how to improve the next iteration. Through external feedback and my own experience organizing the event, the primary pain-point that I wanted to address this year was the burden on hackers for registration logistics and information.

The DeerHacks dashboard aims to consolidate all of the hackathon information, registration logistics, and data management into a single source. We began hiring for tech associates in August and began developing in September. DeerHacks III was planned to take place in the middle of February, earlier than the previous iterations (end of April) so we needed to get the ball rolling as soon as possible. Since we opened registrations in the beginning of January, the tech team had about 4 months to build the full stack.

Development

In the tech team, we worked in 14 day sprints and had daily standups. Along with setting up the DeerHacks Discord server, the four primary services that the team planned on implementing are the following:

Backend Server (Go): Handle REST & RPC API requests and Discord events. Deployed with Docker on AWS EC2

Content Management System (Strapi): Manage and serve schedule and gallery data. Deployed on Digital Ocean

DeerHacks Discord Bot (Python): Sync database and server states for user statuses. Deployed on Digital Ocean

Frontend Dashboard (Nextjs): Event registration, information, and user management. Deployed with Netlify

DeerHacks Tech Team Agile Board

Let's checkout a few features and how they were implemented.

Our team needed to solve the issue of multiple sources of truth. Hackers need to know their registration status and see information that reflects it. Since we received good feedback on the Discord server in previous years, we opted to go with Discord OAuth to handle user authentication. With Discord OAuth, our service is able to get a user's avatar, username, email, and id. ~~Although controversial~~, Discord also allows OAuth apps to join servers on their behalf which allows hackers that sign up for DeerHacks to automatically join the DeerHacks Discord server and have synced user states.

Email for Account Verification

Our team realized that the email linked to a Discord account might not be the email a user wants to use, so we also needed to allow users to change their email. For security, this requires an email verification step in the registration process, so users are required to verify their email before the application form is enabled. We facilitated 3000+ emails with Brevo SMTP for account verification, RSVPs, and newsletters.

Discord Sync

Although Discord OAuth allows our service to have a Discord user's state, we didn't have a way to keep their user state on the DeerHacks Discord server in sync with our service. This would be important as we needed to restrict / enable certain permissions and channels based on a user's status on the dashboard. For example, when a user verifies their email, not only should their permissions / status change on the dashboard but also their Discord user state should also reflect their elevated permissions / status on the DeerHacks Discord server.

We knew that the answer would be for the Discord Bot to handle user update and server join events, but Discord's rate limiting would be something we needed to consider. With only 50 requests per second to work with, 1000+ users to serve, and large timeout penalties, this problem was a high priority to address.

// main.go
func main() {
  // ...

  // Discord Join Queue & Update Role Queue Tasks
  go discord.JoinGuildTask(15 * time.Minute)
  go discord.UpdateRoleTask(10 * time.Minute)

  // ...
}

// discordTasks.go
func JoinGuildTask(interval time.Duration) {
  ticker := time.NewTicker(interval)

  for {
    select {
    case <-ticker.C:
      for {
        // Dequeue Users
        users, err := DequeueUsers("join") // Pops 25 at a time
        if err != nil {
          fmt.Printf("JoinGuildTask - Error dequeuing users: %v\n", err)
          break
        }

        // Process Join Events
        for _, user := range users {
          if user != nil {
            fmt.Printf("JoinGuildTask - Processing user with Discord ID: %s\n", user.DiscordId)
            AddToDiscord(user, false) // Hit Discord API
            time.Sleep(3 * time.Second)
          }
        }
      }

      fmt.Println("Join Guild Task completed", time.Now())

    }
  }
}// main.go
func main() {
  // ...

  // Discord Join Queue & Update Role Queue Tasks
  go discord.JoinGuildTask(15 * time.Minute)
  go discord.UpdateRoleTask(10 * time.Minute)

  // ...
}

// discordTasks.go
func JoinGuildTask(interval time.Duration) {
  ticker := time.NewTicker(interval)

  for {
    select {
    case <-ticker.C:
      for {
        // Dequeue Users
        users, err := DequeueUsers("join") // Pops 25 at a time
        if err != nil {
          fmt.Printf("JoinGuildTask - Error dequeuing users: %v\n", err)
          break
        }

        // Process Join Events
        for _, user := range users {
          if user != nil {
            fmt.Printf("JoinGuildTask - Processing user with Discord ID: %s\n", user.DiscordId)
            AddToDiscord(user, false) // Hit Discord API
            time.Sleep(3 * time.Second)
          }
        }
      }

      fmt.Println("Join Guild Task completed", time.Now())

    }
  }
}

The solution Shiva came up with was to implement an isolated queue for Discord requests. With the DeerHacks database already on AWS RDS, we went with the simple strategy of creating database tables to act as queues instead of using something like Redis or AWS SQS ~~because we're broke bois~~ since Discord updates are "asynchronous" anyways so latency isn't the highest priority (its consistency).

Discord update tasks were handled by Goroutines (we used them kinda like cron jobs). Above is a simplified version of how the server handled Discord update events to keep their user states in sync (full code here). Another bonus of using the Discord Bot is that we could also message users directly on Discord of any status changes or other important notifications.

Registration

The application process should be simple and responsive. The frontend team spent a lot of focus into the application form's accessibility for all types of users. We wanted to develop the dashboard, application form included, to be responsive even on the Samsung Galaxy Fold with a screen width of 280px. For reference, the WCAG 2.1 guidelines specify supporting a minimum of 320px.

We added a Save as Draft button that is enabled when the user updates the form state so that users can revisit their application at a later time, enabling the user to focus on the quality of their application. Similarly, if there were unsaved changes and a user navigates away, the dashboard would display a confirmation modal, preventing any accidental closes.

Hacker Application Form

We used React Hook Form along with Zod on the frontend to validate and control form state. These libraries enabled us to split up form state into multiple sections along with showing and errors or missed fields to the user.

One fun form field to checkout is the resume upload logic on the backend. Here's a simplified / pseudocode version (full code here):

func UpdateResume() {
  // Check Valid Status
  // Check Registration Window Open
  // Get File Data from Request
  // Check Valid File Size & Filename Length
  // Check File is PDF
  // Open File & Compute Hash
  // Early Return if Hash Matches
  // Create S3 Session

  svc := s3.New(sess)

  // Read File Data
  // Check Invalid PDF Data

  // Create S3 Key from User ID
  s3key, err := constructS3Key(user.DiscordId)
  if err != nil {
    c.AbortWithStatus(http.StatusInternalServerError)
    fmt.Println("GetResume - environment not defined for current appEnv", err)
    return
  }

  // Upload File
  _, err = svc.PutObject(&s3.PutObjectInput{
    Bucket: aws.String("dhapplications"),
    Key:    aws.String(s3key),
    Body:   bytes.NewReader(fileData),
  })
  if err != nil {
    c.AbortWithStatus(http.StatusInternalServerError)
    fmt.Println("UpdateResume - Failed to upload file: ", err)
    return
  }

  // Get Presigned URL
  presigned_url, err := getPresignedURL(svc, s3key, file.Filename)
  if err != nil {
    c.AbortWithStatus(http.StatusInternalServerError)
    fmt.Println("UpdateResume - Error in getting presigned url: ", err)
    return
  }

  // Update DB Values
  // Return Resume URL
}func UpdateResume() {
  // Check Valid Status
  // Check Registration Window Open
  // Get File Data from Request
  // Check Valid File Size & Filename Length
  // Check File is PDF
  // Open File & Compute Hash
  // Early Return if Hash Matches
  // Create S3 Session

  svc := s3.New(sess)

  // Read File Data
  // Check Invalid PDF Data

  // Create S3 Key from User ID
  s3key, err := constructS3Key(user.DiscordId)
  if err != nil {
    c.AbortWithStatus(http.StatusInternalServerError)
    fmt.Println("GetResume - environment not defined for current appEnv", err)
    return
  }

  // Upload File
  _, err = svc.PutObject(&s3.PutObjectInput{
    Bucket: aws.String("dhapplications"),
    Key:    aws.String(s3key),
    Body:   bytes.NewReader(fileData),
  })
  if err != nil {
    c.AbortWithStatus(http.StatusInternalServerError)
    fmt.Println("UpdateResume - Failed to upload file: ", err)
    return
  }

  // Get Presigned URL
  presigned_url, err := getPresignedURL(svc, s3key, file.Filename)
  if err != nil {
    c.AbortWithStatus(http.StatusInternalServerError)
    fmt.Println("UpdateResume - Error in getting presigned url: ", err)
    return
  }

  // Update DB Values
  // Return Resume URL
}

By utilizing presigned URLs in AWS S3, we're able to grant users limited time access to a resume's preview URL without compromising on security in the S3 bucket. This is useful for hackers to verify their uploads and for organizers to review resumes without needing to download files.

Organizer Dashboard

Admin Dashboard UI

The DeerHacks dashboard looks different for all user statuses. Volunteers have access to a QR Scanner to scan in hackers for food. Hacker signins require a piece of government issued identification along with the QR code embedded in an accepted hacker's dashboard for organizers to verify identity and age. Organizers have the permissions to scan in hackers for registration along with the ability to manage the user database.

Organizer User Management Table

The users table was designed with the same principles as the registration form. Some features that add to its simple and responsive design are pagination, status filtering, search, and field change highlighting. We decided to include two internal user fields, internal status and internal notes, which are only visible to organizers. The internal status field is used to "commit" a status change before we decide to update a user's "real" status. Internal notes are used by organizers reviewing applications to make any comments on an application.

SELECT hackathon_experience, COUNT(*)
FROM users, applications
WHERE users.discord_id = applications.discord_id
AND users.internal_status = 'selected'
AND applications.is_draft = FALSE
GROUP BY hackathon_experience
ORDER BY COUNT(*) DESC;SELECT hackathon_experience, COUNT(*)
FROM users, applications
WHERE users.discord_id = applications.discord_id
AND users.internal_status = 'selected'
AND applications.is_draft = FALSE
GROUP BY hackathon_experience
ORDER BY COUNT(*) DESC;

With 1000+ applications to review, the organizing team used these conventions to review all applications with the same standard. Another plus to following conventions allows us to perform analytics on user data pre and post hacker acceptances. Above is an example query we used in considering hacker acceptances as an aggregate.

Event Information

From our experience in DeerHacks II, the hackathon schedule is subject to lots of change, especially during the event. In DeerHacks II, I remember committing changes throughout the event just to update the schedule. The schedule felt like it wasn't worth the time to build an interface for, but we also wanted non-technical organizers to be able to update it on the fly.

"There are only two hard things in Computer Science: cache invalidation and naming things."

LISA from Blackpink, probably

Employing Strapi felt like the right decision, enabling organizers to populate and modify event details in an instant. One issue we had with the gallery collection in the CMS is that the image data would reset after a redeployment, so we integrated our gallery uploads with AWS S3 to persist file data.

The DeerHacks schedule and gallery are publicly available, so check 'em out!

D-Day & The Future

So how did DeerHacks III go? I'll leave it up to interpretation with the following image:

Wrong image, I meant this one:

DeerHacks has been many of my firsts; My first hackathon & first place win in 2022, my first time organizing and building a hackathon website in 2023, and my first time leading a dev team to create a hackathon dashboard this year. 2024 also happens to be my last year with this team. DeerHacks' slogan this year is "Dare to Glow" which I initially got from ChatGPT as a "lorem ipsum" to put on the website. However, the team has grown fond of this phrase, embodying the challenge to innovate and illuminate the path for the next generation. I'm excited to see how the next generation dares to glow (cringe, 2024).

👋 Thanks for stopping by!