One of my favorite tools that exists internally is our smart bookmarking tool commonly referred to as “bunnylol.” Originally named bunny1 and open sourced, it’s what we use to navigate across all tools, wikis, knowledge bases, and everything else one might use working at Facebook.
An example of a bookmark I use often is the cal command. I can type in the browser cal and it will take me to our internal calendar tool. Another example is the `wiki` command, which I can use to search our internal wiki pages. I type in `wiki` followed by the name of something I’m looking for and it will provide search results for wiki pages that match what I’m looking for.
As you can imagine, it’s immensely helpful! It also provides a way for us to not only bookmark things like the calendar, but also do things like search wikis using the provided queries. Talk about smart bookmarks, eh?
Today, I’m going to show you how to build a simple clone of bunny1 using Rust and Rocket (a web-framework for Rust). The original implementation is written in Python as a web server. We’re going with Rust because it’s:
Let’s get to it!
Note: if you prefer learning by watching, you can follow along in the video series on the Facebook Open Source YouTube channel.
Before building the app, I’ll cover how it will work and the prerequisites for following along. After that, I’ll jump in and go through each step before setting it up for production.
How will it work?
Starting from a high-level, the app will work something like this.
Say for instance I was using Firefox and I had a command gh <page/> which redirected to a page on GitHub like a repository. It would look like this:
Here is that flow in action:
I want to dig more into the technical details of this approach. I can achieve this by building a basic web server app that listens for requests and redirects them based on if it matches specific criteria. I am not building a client-side app because the only functionality I need is a redirect so it makes more sense to be built as a web server.
Using the same example, the application logic will flow like this:
I will use a custom search engine which will allow me to connect the address bar to my own search engine. With the basic flow down, I am ready to move on to the next step!
In order to start writing the Rust application, I will need to do a few things:
I will walk through each of these.
Install Rust
To install Rust, I will use a tool called rustup. You can install it by running the command:
shell curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
This will install rustup which is used for installing Rust and staying up to date with different versions of the language. After you run it and follow the instructions, you should have Rust installed.
You can double-check by running:
shell cargo --version
Cargo is the official Rust package manager and we will use it to compile, build and run our Rust code.
Switch to Nightly Rust
Rust has three release channels:
The stable build is what most people use. I am using the nightly version because Rocket, the framework I’m using, uses “Rust's syntax extensions and other advanced, unstable features.”
You can change this by running the command:
shell rustup default nightly
You can always change it back to the stable version by running:
shell rustup default stable
Set up VS Code for Rust
For this walkthrough, I suggest using VS Code as an editor.
Make sure to install the Rust(rls) extension, which will make it a lot easier to build the app thanks to code completion and other handy features.
If you’re looking to add the extension in another editor, you can take a look at the rust-lang website to see if it’s supported.
Test Rocket with Hello World
To make sure you’re ready to develop with Rocket, I’ll walk you through their “Hello world” getting started application.
Create a new project and cd into it by running:
shell cargo new rusty-bunny --bin cd rusty-bunny
Add Rocket as a dependency to your Cargo.toml like so:
diff [dependencies] + rocket = "0.4.4"
Modify the src/main.rs so that it looks like this:
rust #![feature(proc_macro_hygiene, decl_macro)] #[macro_use] extern crate rocket; #[get("/")] fn index() -> &'static str { "Hello, world!" } fn main() { rocket::ignite().mount("/", routes![index]).launch(); }
Inside the hello-rocket directory, run the command:
shell cargo run
This will compile and run your application. Verify that it’s working by opening up http://localhost:8000/ in your browser. You should see the text “Hello, world!”
I’ll reuse this repo for the project so I don’t have to create a new one.
The first thing I’m going to do is set up the search route. Inside src/main.rs, add the following block:
rust #[get("/search")] fn search() -> &'static str { "Hello from the search page!" }
I copied this from the root route (“/”). Before I can test, I need to inform Rocket that there is a new route. Near the bottom, make the following adjustments and add “search” to the array after “index”:
diff fn main() { + rocket::ignite().mount("/", routes![index, search]).launch(); }
Now, Rocket knows to set up two routes for you: index and search. Open up http://localhost:8000/search and verify that it shows our message “Hello from the search page!”.
Great!
I need to do two more things before I’m done here.
To allow query params, I need to modify the route path. Make the following changes inside src/main.rs:
diff +#[get("/search?<cmd>")] +fn search(cmd: String) -> &'static str { + println!("You typed in: {}", cmd); "Hello from the search page!" }
Here, I made the following changes:
This means when you visit http://localhost:8000/search?cmd=tw, you will see “You typed in: tw” logged to your terminal console (where you ran `cargo run`). Go ahead and try it out!
Now that I have the search route ready that accepts query params, I can move on to the next step.
The next piece of logic that I’m going to add to the application is parsing the command from the query string and then redirecting based on that string.
Parsing Query String
I will set up our logic so that it grabs all the characters up to the first whitespace in the query string and uses that as the command.
To start this off, go ahead and write a unit test using Rust’s built-in testing features. At the bottom of src/main.rs, add the following code:
rust fn get_command_from_query_string(query_string: &str) -> &str { let test = "hello"; return &test; } #[cfg(test)] mod tests { use super::*; #[test] fn test_get_command_from_query_string_no_whitespace() { // Test with command only let actual = get_command_from_query_string("tw"); let expected = "tw"; assert_eq!(actual, expected); } #[test] fn test_get_command_from_query_string_with_whitespace() { let actual = get_command_from_query_string ("tw @fbOpenSource"); let expected = "tw"; assert_eq!(actual, expected); } }
I first add a function called get_command_from_query_string which I will use to parse the query string. For now, it always returns the word “hello” along with two unit tests for the function. You can read more about unit testing in the Rust By Example Book Unit testing chapter.
The tests will ensure I can get the command when I have a query string with whitespace and without. This will ensure I can use commands that take arguments such as navigating to a user’s Twitter profile page.
To run this and watch it fail, run cargo test in your terminal.
Now, I’m going to modify the function so the tests pass.
diff fn get_command_from_query_string(query_string: &str) -> &str { - let test = "hello"; - return &test; + if query_string.contains(' ') { + // We need to this to know where to slice the string + let index_of_space = query_string.find(' ').unwrap_or(0); + return &query_string[..index_of_space]; + } + // Otherwise, return the query string as is + &query_string }
In the logic, it checks the query string for a whitespace character using the .contains method. If it does contain one, it finds the index of the white space and returns the query string from the beginning up to the whitespace character using [...index_of_space]. If it doesn't find a whitespace, then it assumes that the query string is the command and returns as is.
Let’s re-run our tests and see if they pass. And they do. Woohoo! The first real logic and tests are now in place.
The next step is to modify the search function to allow for redirects. Once completed, I want it to redirect based on the command parsed from the query string. Working in small chunks, I’ll split this in two steps:
Near the top of the file, import the Redirect struct from rocket::response like so:
diff #![feature(proc_macro_hygiene, decl_macro)] #[macro_use] extern crate rocket; + use rocket::response::Redirect; #[get("/")] // ...
Then modify the search route return type and return a basic Redirect to https://www.google.com/.
diff #[get("/search?<cmd>")] + fn search(cmd: String) -> Redirect { println!("You typed in: {}", cmd); - "Hello from the search page!" + let redirect_url = "https://google.com"; + Redirect::to(redirect_url) }
Test it in your browser by restarting the server and running cargo run and navigate to http://localhost:8000/search?cmd=hello. If it redirects you, then it worked!
The last step is to add the function I created to parse the command, and then redirect based on what it gets. I’ll add that in.
diff #[get("/search?<cmd>")] fn search(cmd: String) -> Redirect { println!("You typed in: {}", cmd); + let command = get_command_from_query_string(&cmd); + let redirect_url = match command { + "tw" => String::from("https://twitter.com"), + _ => String::from("https://google.com") + }; Redirect::to(redirect_url) }
First, it parses the command from the query string. Then, the app utilizes Rust’s pattern matching feature using the match keyword. To test things out, I add the “tw” as one of my commands to redirect to Twitter. If there are no matches, it simply redirects to Google. Notice, I am not yet using the query string arguments. Eventually, I will refactor this and take the query string and use it with the redirect so that it performs a Google search for the user.
Go ahead and test it out by visiting http://localhost:8000/search?cmd=tw. Don’t forget to restart your server!
With that, I now have the first command up and going! Awesome!
Next up in the application is constructing smart redirects for sites like GitHub, Twitter and Google. I’m also going to make use of modules to keep the code clean, cozy and organized. As usual, I want to take this step-by-step in the following order:
Creating a utils Module
According to the Rust By Example Book, the module system allows you to “split code in logical units (modules), and manage visibility (public/private) between them.”
I’m going to create the first module by creating a utils directory inside src. Next, create a file called mod.rs inside src/utils. This will be the utils module. Inside this file, copy and paste the function get_command_from_query_string along with it’s tests.
Due to the private-by-default nature of modules in Rust, I need to add the keyword pub in front of the function like so:
diff + pub fn get_command_from_query_string(query_string: &str) -> &str { //...
Heading back into the src/main.rs, I can now utilize this module by making two minor adjustments:
And that’s it! Everything should still work. I have the first module. The nice part about this is you can see the name of the module, utils, lines up with the name of the directory. This makes it easier to discern where the code lives in the codebase.
Add Smart Google Redirect
Remember how I said I wanted to use the query string in a Google search if it didn’t match any of the known commands? Now is the time where Iadd this in.
Continuing with the modules, I’m going to create a new file called google.rs under src/utils. Inside this file, I’ll add the function declaration and two tests:
rust pub fn construct_google_search_url(query: &str) -> String { // TODO add logic String::from("https://google.com/search?q=test") } #[cfg(test)] mod tests { use super::*; #[test] fn test_construct_google_search_url() { let fake_query = "hello"; assert_eq!( construct_google_search_url(fake_query), "https://google.com/search?q=hello" ); } #[test] fn test_construct_google_search_url_with_encoding() { let fake_query = "hello world"; assert_eq!( construct_google_search_url(fake_query), "https://google.com/search?q=hello%20world" ); } }
Before I can run our test, Cargo needs to find it, which means it needs to be referenced in a main.rs or from a sub-module used in main.rs. Inside src/utils/mod.rs, add the line pub mod google; to the top. This includes the google modules with our utils module so that Rust knows it’s used in the program and will include the tests.
Running cargo test, you should see it fail. Next, I’ll make it pass!
The first thing I need to do is add a dependency which will encode the query string and pass the second test. Inside your Cargo.toml, add the following:
diff [dependencies] rocket = "0.4.4" +percent-encoding = "2.1.0"
The percent-encoding crate will do the heavy lifting of the encoding for you. Now, make the following changes to src/utils/google.rs:
diff + extern crate percent_encoding; + use percent_encoding::{utf8_percent_encode, AsciiSet, CONTROLS}; // Used as part of the percent_encoding library + const FRAGMENT: &AsciiSet = &CONTROLS.add(b' ').add(b'"') .add(b'<').add(b'>').add(b'`'); pub fn construct_google_search_url(query: &str) -> String { - String::from("https://google.com/search?q=test" + let encoded_query = utf8_percent_encode(query, FRAGMENT) .to_string(); + let google_search_url = format!( "https://google.com/search?q={}", encoded_query); + google_search_url }
I made some significant changes. First, I used the extern keyword at the top to let Rust know that I’m using an external dependency in this module. Then, I imported a few utilities from the crate.
Next, I used the const to declare a new variable. I did that so that Rust can work its magic now that it knows the value will never change (hence “const” for “constant”). As you can see from the type, it’s an AsciiSet. I won’t dive down the rabbit hole here. Know that it’s extending the provided CONTROLS AsciiSet and adding the following characters: whitespace, double-quote, angled brackets and the back-tick.
Inside the functions, I have two new let`s: one for the encoded_query and one for the google_search_url. It takes in the query as a parameter, encodes it, and then returns it to the function caller.
Now I want to see if the tests pass by running cargo test. And they do! Woohoo!
I’ll head back over to src/main.rs where I’ll add in the new utility function:
diff #[get("/search?<cmd>")] fn search(cmd: String) -> Redirect { println!("You typed in: {}", cmd); let command = utils::get_command_from_query_string(&cmd); let redirect_url = match command { "tw" => String::from("https://twitter.com"), + _ => utils::google::construct_google_search_url(&cmd) }; Redirect::to(redirect_url) }
Since the google module is nested within the utils module`, you access it by calling utils::google::construct_google_search_url and passing in a reference to the search query `cmd`.
Run cargo run in your terminal and navigate to http://localhost:8000/search?cmd=hello%20world
It should redirect you to Google and show the search results for “hello world”. Woot woot! Onward.
With this pattern in place of writing modules for the smart redirects, I’m going to add two more: Twitter and GitHub. I’ll write tests and then add functionality for each. I’ll start with Twitter.
For Twitter, I want to be able to do the following:
Go ahead and create a new module at src/utils/twitter.rs. Add the following code and tests:
rust pub fn construct_twitter_url(query: &str) -> String { // fill in logic String::from("Hello world") } pub fn construct_twitter_profile_url(profile: &str) -> String { // fill in logic String::from("Hello world") } pub fn construct_twitter_search_url(query: &str) -> String { // fill in logic String::from("Hello world") } #[cfg(test)] mod tests { use super::*; #[test] fn test_construct_twitter_url() { let fake_query = "tw"; assert_eq!(construct_twitter_url(fake_query), "https://twitter.com"); } #[test] fn test_construct_twitter_url_query() { let fake_query = "tw hello world"; assert_eq!(construct_twitter_url(fake_query), "https://twitter.com/search?q=hello%20world"); } #[test] fn test_construct_twitter_url_profile() { let fake_query = "tw @fbOpenSource"; assert_eq!(construct_twitter_url(fake_query), "https://twitter.com/fbOpenSource"); } #[test] fn test_construct_twitter_profile_url() { let fake_profile = "jsjoeio"; assert_eq!( construct_twitter_profile_url(fake_profile), "https://twitter.com/jsjoeio" ); } #[test] fn test_construct_twitter_search_url() { let fake_query = "hello world"; assert_eq!( construct_twitter_search_url(fake_query), "https://twitter.com/search?q=hello%20world" ); } }
You’ll notice I added three functions and some tests
I want to expose our module in mod.rs so that I can run the tests and watch them fail.
diff pub mod google; + pub mod twitter;
Run cargo test and see the tests all fail. Great. I’m ready to make them pass!
Working backwards, I’ll start with the construct_twitter_search_url since it is most similar to the smart Google redirect. Inside src/utils.twitter.rs, make the following changes:
diff +extern crate percent_encoding; +use percent_encoding::{utf8_percent_encode, AsciiSet, CONTROLS}; // Used as part of the percent_encoding library + const FRAGMENT: &AsciiSet = &CONTROLS.add(b' ').add(b'"') .add(b'<').add(b'>').add(b'`'); //… pub fn construct_twitter_search_url(query: <str) -> String { - String::from("Hello world" + let encoded_query = utf8_percent_encode(query, FRAGMENT).to_string(); + let twitter_search_url = format!("https://twitter.com/search?q={}", encoded_query); + twitter_search_url }
And now I can run our test for this function by running:
shell cargo test -- test_construct_twitter_search_url
The -- tells cargo to look for exactly this test and this test only. Great, that passes. Next up, I want to tackle the construct_twitter_profile_url. This one is straightforward. It takes in the profile and then appends it to https://twitter.com/ to visit the profile page.
diff pub fn construct_twitter_profile_url(profile: &str) -> String { - String::from("Hello world") + format!("https://twitter.com/{}", profile) }
The logic related to parsing the username from tw @username will come in the next function. Here it only needs to replace the {} with profile and then return the string. I use the format! macro provided by Rust. Now, I’m going to run this exact test by running:
shell cargo test -- test_construct_twitter_profile_url
Finally, I’m going to add the logic for the main construct_twitter_url function. Modify the function to match this and then I’ll go over what’s happening:
diff pub fn construct_twitter_url(query: &str) -> String { - String::from("Hello world") + if query == "tw" { + let twitter_dotcom = "https://twitter.com"; + + twitter_dotcom.to_string() + + // Check if it looks like a Twitter profile + } else if &query[..4] == "tw @" { + construct_twitter_profile_url(&query[4..]) + } else { + // Assume the other match is "tw sometext" + // and search on Twitter + construct_twitter_search_url(&query[3..]) } }
First, it checks if the query string matches “tw” meaning the user entered “tw” with no other text. If so it sends them to the Twitter homepage. Otherwise, it checks if the first 4 characters match “tw @”. If it does, then it constructs a twitter profile url. The final alternative is taking the text and using it in a search query.
I’m going to run the tests to make sure things are working as expected using cargo test. And they do! Great. The last step is to add this to the main function.
diff let redirect_url = match command { + "tw" => utils::twitter::construct_twitter_url(&cmd), _ => utils::google::construct_google_search_url(&cmd) };
You know the drill! Run cargo run and try out a few scenarios:
As long as all are working, I can move onto GitHub.
I won’t walk through all the same details for GitHub. Instead, I’ll show you the solution and then talk through what’s happening. It’s mostly similar to what I did previously.
Create a new file src/utils/github.rs and add the following contents:
rust extern crate percent_encoding; use percent_encoding::{utf8_percent_encode, AsciiSet, CONTROLS}; // Used as part of the percent_encoding library const FRAGMENT: &AsciiSet = &CONTROLS.add(b' ').add(b'"').add(b'<') .add(b'>').add(b'`'); pub fn construct_github_url(query: &str) -> String { if query == "gh" { let github_dotcom = "https://github.com"; github_dotcom.to_string() } else { // Assume the other match is "gh page" let encoded_query = utf8_percent_encode(&query[3..], FRAGMENT) .to_string(); let github_url = format!("https://github.com/{}", encoded_query); github_url } } #[cfg(test)] mod tests { use super::*; #[test] fn test_construct_github_profile_url_with_gh() { let fake_query = "gh"; assert_eq!(construct_github_url(fake_query), "https://github.com"); } #[test] fn test_construct_github_profile_url_with_repo_url() { let fake_query = "gh facebook"; assert_eq!( construct_github_url(fake_query), "https://github.com/facebook" ); } #[test] fn test_construct_github_search_url_with_repo_url() { let fake_query = "gh facebook/docusaurus"; assert_eq!( construct_github_url(fake_query), "https://github.com/facebook/docusaurus" ); } }
This smart redirect supports two cases:
First, it checks if the query string matches “gh” meaning that’s all the user gave the program. If so, it redirects to GitHub’s home page. If not, it optimistically assumes they entered a page such as <username> or <username/repository>.
I have two tests to make sure things work as expected. I need to add the module to src/mod.rs with pub mod github;. Running these with cargo test, the tests pass and I can ensure the functions are working properly.
Last, I add this to the src/main.rs so I can actually use it:
diff let redirect_url = match command { + "gh" => utils::github::construct_github_url(&cmd), "tw" => utils::twitter::construct_twitter_url(&cmd), _ => utils::google::construct_google_search_url(&cmd) };
Test it out by running cargo run. Try the following:
Woohoo! I now have smart Google, Twitter and GitHub redirects!
Up until this point, I have been testing this out by navigating to the url directly. Before I deploy the app to production, I am going to test it locally using an actual custom search engine in both Firefox and Chrome.
Testing with Firefox
To set up the new application to be used as a custom search engine in Firefox, you can follow these steps:
*Note: if you forgot to check the box, you can change the search engine by navigating to about:preferences#search
If your app isn’t already running locally, run cargo run.
Open a new tab in Firefox and try some of the commands we added! With that, the app is working locally! Party-time!
Testing with Chrome
Setting up with Chrome is a little bit more straightforward because you don’t need any extensions or add-ons. Follow these steps:
Open a new tab and test out the commands. Woot woot! It is working in both Firefox and Chrome. High-five to self!
To deploy the app and be able to use it in production, I’m going to deploy it to Heroku and then update the custom search engines on Firefox and Chrome.
Deploying to Heroku
If you don’t have an account, go ahead and create a free Heroku account. Thankfully, Heroku’s “Free and Hobby” plan is enough for this app’s usage. It means the app will have a cold start because they put the dynos to sleep, but fast after the first request.
After that, follow these steps to deploy your app:
/target/* !/target/release
You should be good to go! Navigate to the Heroku dashboard and find your project’s URL.
I also recommend setting up automatic deploys with GitHub or another version control platform. I do this for my own projects and it allows me to make changes and push to GitHub, and then Heroku deploys a new app automatically.
Note: if you run into any strange issues here, check out this GitHub issue which may help you troubleshoot them.
Installing on Firefox or Chrome
Since I already followed the steps above to use custom search engines on Firefox and Chrome, all I need to do is go in and edit the URLs. It also may be easier to delete the search engines I set up and start them over again.
Instead of using “http://localhost:8000/search?cmd=%s”, I’ll want to update it to match my new Heroku app URL.
Test out the app with a few commands and rejoice as it works in production!
Congratulations on making it through this project with me! It was a lot. I built a smart bookmarking tool using Rust and the web server framework Rocket. The app serves as a custom search engine which can be used with both Firefox and Chrome. I added commands for Twitter and GitHub and set up a redirect to a Google search for things that didn’t match the predefined commands.
If you’d like to see a full-working version of this, you can check it out here on GitHub.
What’s Next?
If you’d like to continue hacking on this project further, here are a few ideas:
Let us know if you do this or something similar! We’d love to see what you build.
Resources for Learning More Rust
Looking for more ways to develop your Rust skills? Here are a few recommendations for continuing your learning:
Thanks for reading! Happy coding!
To learn more about Facebook Open Source, visit our open source site, subscribe on Youtube, or follow us on Twitter and Facebook.
Interested in working with open source at Facebook? Check out our open source-related openings on our career page by taking this quick survey.