My Experience Contributing to an Open Source Project

About the Project

Firefiles is an open-source software that serves as an alternative to Dropbox, allowing users to manage their files across various cloud storage providers. It provides a file system interface for these providers, giving users control and privacy over their stored files.

With Firefiles, users have more control over their data and the cloud provider they use. This is particularly important for those who want to avoid vendor lock-in and have the ability to switch providers easily. Firefiles enables users to choose their preferred cloud storage provider based on factors like pricing, features, and terms of service. This flexibility is crucial for individuals and businesses looking to optimize their storage solutions while avoiding the limitations of a one-size-fits-all approach.

Developers who need to manage and share code repositories, project files, or other developmental assets may find Firefiles valuable. It allows them to use cloud providers that align with their development workflows, and the unified file system interface can simplify collaboration among team members. Users who want to migrate away from services like Dropbox or Google Drive due to pricing concerns, storage limitations, or terms of service might choose Firefiles. It offers a more flexible pricing model and the ability to leverage the unique features of different cloud providers.

The Issue

The issue being addressed here for Firefiles is the integration of the Wasabi and Digital Ocean object storage platforms into the existing system. The goal is to enhance Firefiles' capabilities by adding support for Wasabi as an additional cloud storage provider alongside Backblaze and AWS S3. This integration allows users of Firefiles to efficiently manage their files using the two storage services. Due to their shared similarities, the implementation of Wasabi is identical to that of Digital Ocean

Different users have different preferences and requirements when it comes to cloud storage providers. By offering support for multiple providers, Firefiles allows users to choose the one that best fits their needs, whether it's based on pricing, features, performance, or data residency.

Our complete methodology for implementing Wasabi and Digital cloud providers are as followed:

1. Create a new file within the pages/new/ directory. This file will be responsible for obtaining the necessary credentials for the provider. Similar files for other providers can serve as a reference for implementing this.
2. Modify the useS3 hook to make it compatible with the provider. This involves making necessary changes to the hook's logic to ensure that it can communicate effectively with the cloud storage.

1. Add an entry for the provider to the PROVIDERS array in the util/globals.ts file. This step ensures that the Firefiles app recognizes Wasabi or Digital Ocean as a valid cloud storage option.
2. Add the name of the provider to the Provider enum in the util/types.ts file. This ensures consistent handling and identification of the provider throughout the app's codebase.
3. In the useBucket hook, add a case specifically for the provider. This case should return the appropriate hook implementation for interacting with the cloud storage.
4. Adjust any provider-specific functions in the codebase to ensure they work seamlessly with the newly integrated provider. This involves modifying conditional statements or switch cases that differentiate between different providers.

Codebase Overview

Firefiles utilizes typical technology stack for a web-based file management and cloud storage application, which includes:

1. Vercel: a cloud platform designed for deploying and hosting web applications, particularly those built with frontend frameworks like React and Next.js. It provides a simple and efficient way to deploy, manage, and scale web projects.
2. Next.js: a popular React framework that simplifies the creation of server-rendered and statically-generated React applications. It offers features like server-side rendering (SSR), static site generation (SSG), routing, and more, making it easier to build performant and SEO-friendly web applications.
3. React: a JavaScript library developed by Facebook for building user interfaces (UIs). It allows developers to create reusable UI components and manage their state efficiently. React uses a declarative approach, enabling developers to describe how their UI should look at any given state, and it automatically updates the UI when the state changes.
4. JavaScript: a widely used programming language for building dynamic and interactive web applications. It's primarily executed in the browser, enabling developers to create user experiences that respond to user input and events. JavaScript is essential for frontend web development and is often used alongside HTML and CSS.
5. TypeScript: a superset of JavaScript that adds static typing to the language. It allows developers to define explicit data types for variables, function parameters, and return values. TypeScript helps catch errors and provides better tooling support, making code more robust and easier to maintain, especially in larger projects.

To write our code, we copy the parent source code of Firefiles into a repository on GitHub, and perform our shared contribution through GitHub Codespace, a cloud-based development environment offered by GitHub, designed to simplify the process of coding, collaborating, and contributing to projects.

Technical Challenges

There are two most challenging roadblocks we faced during this project. One is the deployment of Firefiles in our development server. In order to test the connection of the cloud providers, we need to deploy the application within our local port, however, in the early days, we struggled to correctly define the environmental variables needed to successfully deploy the application.

Another problem we faced toward the end is fetching the data from the cloud providers to Firefiles. By default, the cloud providers that we attempted to integrate possess a CORS policy that restricts access points to certain URLs that need to be manually entered. We didn’t know this at first and were frustrated at how Firefiles is unable to create a new cloud drive in our server no matter how many times we revise the code.

The first challenge is the more simple of the two. For unknown reasons, the command provided in the application documentation to start the development server “pnpm dev” is not useful in our local machines, so we had to research another command “pnpm start” to start the application. Additionally, prior to deployment, a set of environmental variables prescribed in the documentation had to be built properly for the server to start.

However, the second challenge is much more sophisticated and was not obvious to us at first, because it requires changes not inside the source code, but inside the policy of the cloud provider buckets. Although there are no error within the code, Firefiles is unable to fetch data to a new drive of the cloud provider, a persistent CORS error causes the drive creation process to immediately halt. Upon further research, we found that by editing the CORS permission of the cloud drive to include all referred endpoints, including our local own, we can bypass the CORS error, which is meant to increase security to the data.

Regardless of challenges, throughout the process, our mentor is closely in touch with our challenges with weekly meetings and occasional progress review and recommendation. We also reached out to the owner in charge of Firefiles for advice, he was very friendly and helpful and explained the process carefully.

The technical difficulties are now largely solved. However, a merge conflict is discovered that is preventing our code to be correctly deployed when merging with the original repository, but it is outside the range of time and our assigned work.

Solution

The primary issue we are assigned to is to integrate the cloud storage platform Wasabi into the Firefiles API. According to the instructions, Wasabi and Digital Ocean fully support the AWS S3 API that is already implemented within the software, additionally, the code supporting AWS S3 and Backblaze, a newly integrated storage platform similar to the two we are assigned to, hence, Wasabi and Digital Ocean integration can be replicated using working templates of the S3 and Backblaze hooks without "reinventing the wheel again." Initially, we only approached Wasabi, and only implemented Digital Ocean toward the end in a similar technique only when this technique is confirmed by us to be working.

We have followed and completed the instructions closely. First, we created a new file inside pages/new/ directory which is used to prompt the user to get the credentials for the Wasabi bucket, as well as creating the bucket itself, which we call wasabi.tsx and is similar to backblaze.tsx in the same directory. Second, we made changes in the useS3.tsx hook in the web/hook/ directory, this file contains the main functionalities of the application that is object management. We created a new condition that checks if the bucket provider is Wasabi to establish connection, again, we applied the previous conditions used by S3 and Backblaze. Third, we made changes in the useBucket.tsx to include a function that uses the S3 connection if the provider is Wasabi. In conjunction to these, the Wasabi provider name has also been added to the arrays in globals.ts and types.ts in web/util/. This can be reflected by our concise work due to our experience with JavaScript and TypeScript.