-
Notifications
You must be signed in to change notification settings - Fork 0
Final Report
A final report report.pdf (5-6 pages) that describes how well the project worked out. How did your milestones go? What was your experience with design, interfaces, testing, etc.? What surprises, pleasant or otherwise, did you encounter on the way? What choices did you make that worked out well or badly? What would you like to do if there were more time? How would you do things differently next time? What should next year's class learn from your experience? Imagine that you are writing this document for your (friendly!) boss or professor and want to explain what you learned that could be applied to the next project.
Grading for report: real insight, thoughtful, informative / straightforward "this is what we did" / repetitive; uncoordinated sections / little evidence of thought or effort
Our planning was successful in that we accurately predicted the sequence of items we would work on and which features we would prioritize. However, difficulties with setting up the project at the start (specifically, failed attempts at using Revel as an http framework for the backend, and various React starter kits for the frontend, as well as the generally high startup costs of Go and React-Redux compared to other architecture options) put us a week behind our intended development schedule. From there, we generally stayed consistently a week behind our original intended schedule.
Rather than concretely design the frontend of the app from the very beginning, we followed an agile development process. We started out with the most basic features of the app laid out in the simplest, most obvious ways possible (e.g. the very first iteration of our app simply displayed a list of the most recent listings) and iterated from there. We took many design cues from Google Inbox, including the list of expanding cards, the position of the search bar, and the layout of the navigation menu. We believe this was a reasonable and sustainable way to design the UI--we managed to keep the app layout coherent as we added features, without creating large gaps in the UI where missing features would have been, especially given that none of us have meaningful experience in UX design.
The database, in contrast, was considerably less volatile, in part due to the nature of relational databases making it difficult to make frequent changes. At the start of spring break we worked out exactly what data we needed for the functionality we included in our design document and set up our schema accordingly. Because of our initial thoroughness, our schema remained unmodified for most of the project, with the only changes being adding some minor constraints later on and incorporating string arrays once we realized they were possible and made our code simpler. The backend was somewhat volatile, with frequent tweaking and refactoring, but few major rewrites or changing to existing code.
Automated Testing: From the start, we set up automated testing, continuous integration, and code coverage analysis, both on the frontend and the backend. However, this largely fell by the wayside as we began to realize that trying to learn automated testing took time away from writing code. As a result, we gradually transitioned to using fewer and fewer tests, writing stubs, and eventually not using our automated testing. This runs against conventional wisdom, but we were in a time crunch and so couldn't justify automated testing over the thorough manual testing we would need to do anyway.
User Testing: We asked friends to perform a series of simple tasks on our website according to a short script we made, which can be seen here. Examples included asking them to buy or sell items that were or were not already listed. User testing helped us identify confusing names and inconsistencies that we fixed. Our website includes a feedback link for bug reports and general user experience, but we did not get any responses. Furthermore, we posted to the old TigerTrade asking for current users to try out our website, but we did not get any response from there either.
Consulting with GSG: Jon Balkind helpfully offered to put us in touch with the graduate student government. With his help, we arranged to meet with the GSG and talked to them about their impression of the old TigerTrade implementation and what they wanted from a new version. Jon also tried out a later version of our site and gave feedback on the interface and how it compared to the old TigerTrade.
React and Redux turned out to be excellent choices for managing the views of our app. React allowed us to write all of the display code as functions of data stored in the application store, minimizing the potential for weird UI bugs and making it much easier to debug unexpected behavior. If something went wrong, we could check the values in the store; if those were correct, we could then focus our attention on how our view code processed the data in the store for display.
While Redux required more boilerplate at the start compared to other ways we could have managed application state, it quickly became clear that the benefits of the Redux pattern--unidirectional data flow from store down through the component hierarchy, and only modifying the state of the app through dispatching actions to the reducers--were well worth the startup costs.
The Material-UI library was a mixed bag. At the start, it handled a lot of styling work for us, and it put Evan's Android development experience to good use when it came to laying out the app. It also served as a helpful constraint on design choices--with Google's Material Design guidelines to follow, we avoided wasting time arguing over design. However, the extreme reluctance of the Material-UI library maintainers to accept pull requests, and the general lack of customizability and composability of many of the library components, ended up being somewhat frustrating and even pushed us to fork our own version of Material-UI in order to be able to make small modifications to the DatePicker component.
Redux-Form was probably our most disappointing library decision on the frontend. The library appears to be very mature, but the way the reduxForm decorator connected form inputs to the store was imperfectly documented. If we were able to do the project over again, we would write code for handling the form state ourselves. Overall, the redux-form component API turned out to be more restrictive than convenient, with strange side effects on styling.
One of the most impactful choices for the backend of the project was the decision to use Go. We made this choice not due to any technical reasons, but because we all had been meaning to learn Go for a while and saw this as an excellent opportunity to do so. Choosing our tools before we had decided on what our project was going to be, predictably, caused problems when our tools turned out not to be the best fit.
Here are some of the issues we ran into in particular:
- Libraries to accomplish fairly common tasks were not as supported as in other languages-- especially CAS authentication, where we had to manually fork and modify the CAS package to make it work. Poor documentation, unmaintained packages, and lack of a cohesive community were all issues that we encountered throughout the project.
- Dependency management under Go made configuration difficult. Go expects the code to be in a very specific place, so renaming the folder that the project was contained in or changing the owner of the github repository broke our code. Eventually, we converged on using govendor, but the lack of a community-sanctioned package management system proved frustrating.
- Testing proved difficult as well, since Go doesn't have the language-level support of monkey-patching and metaprogramming that, say, Rails does. So setup was explicit, teardown was explicit, and the tests we wrote were verbose. This, among other things, prevented us from writing more automated tests for TigerTrade.
However, not to be so pessimistic, we did find that Go had some nice properties as well:
- When implementing worker-type processes, Goroutines proved immensely useful. In Django, for example, we would have had to install Celery, get RabbitMQ setup, etc. to use worker processes, but Go let us define and run worker-type functions with virtually no performance hit.
- We really understood everything our code does. Because Go abhors the "magic" present in other languages, everything that happened did so explicitly, and there was no framework to magically give us CORS headers, CSRF, or SQL queries. So we wrote it all out ourselves, and as a result have a deeper understanding of what makes a web application actually tick.
Perhaps the biggest mistake we made on the backend was initially deciding to use the Revel web framework for Go. Revel describes itself as a "batteries-included" web framework. As it turned out, Revel was supplying a car battery when we needed a watch battery. It was designed to be used in ways that we did not want to use it, and we spent far more time working against the framework than with it. After a week of struggling, we decided to scrap it and start over with a series of smaller, more specific libraries that did no more than what we needed. We lost an entire week of work on the backend due to this, and remained roughly a week behind for the duration. (We never fell any more behind than that and if it weren't for this mistake we would have been on schedule the entire time, so our initial estimates were precise, if not accurate.)
Ultimately, while we do feel that Go worked a lot against us, we also realize that it was simply not the tool for the job. Go works incredibly well when performance or security is critical, since it allows the developer to understand the code very well. Go is also a great choice when doing network programming and advanced concurrency, as long as the developer can take time to find the optimal solution. However, in a 333 project, performance is largely irrelevant (no user will care that our Google Pagespeed score is 99/100, for example) and rapid development is key instead. Go does not prioritize this usage.
We used PostgreSQL because it was the most supported relational database for Heroku, which we used for hosting. Perry and Maryam did most of the database related work over spring break. Most of our experience was with MySQL, yet we later learned that PostgreSQL supported several features that MySQL lacked. In particular, PostgreSQL supports arrays, which simplified our schema for keeping track of query keywords and listing photos. We also later realized that PostgreSQL supports full text semantic searching. Some features, such as Postgres's excellent search, were not discovered until after we had already written our own solutions in Go, simply because we didn't realize Postgres was that powerful.
Our entire team has past experience with using Git for version management and many of our members had experience using it in real-world environments. This allowed us to work concurrently through effective use of branches. Also, extensive use of pull requests for code reviews meant that every line of code in our project was read and understood by at least one other team member. This helped us avoid accumulating too much technical debt.
We were able to eschew all manner of pointless arguments about code style by adopting at the outset of our project a set of widely-used standards for code formatting. (Our backend code is checked by gofmt and golint and our frontend follows Airbnb's eslint specifications.) Both of these tools have strong editor integration and enforced good coding habits. As a result, we believe that our code is standardized enough to be very readable for future maintainers of the project.
Our product supports the key features of the two alternative platforms (the old version of TigerTrade and the Free and For Sale Facebook group), as well as other features that were requested by current graduate student users.
Better UI: Our UI allows users to browse through items fluidly. We support infinite scrolling and smoothly integrate images on the same page as listings. This is an improved browsing experience over old TigerTrade's truncated pagination and lack of easy access to images without opening a listing, as well as Free and For Sale's structureless compilation of items.
Low-friction contact: Users are notified via an email about potential interest in their items or their buy requests, which can be sent using one click and can include an optional message. This removes the barrier of having to open a Facebook Messenger window (in the case of Free and For Sale) or an email client (in the case of old TigerTrade) to contact buyers or sellers.
Buy requests: Our product allows users to request to buy items that are not currently listed. This feature exists in the alternative platforms but is rarely used, despite current users' expressing interest in its inclusion. Our implementation makes the distinction between buy and sell requests very clear, and allows each category to be composed and browsed separately.
Instant, semantic search: Unlike the alternative platforms, which only search for exact word matches, our application supports semantic and fuzzy search. This means a listing including the word "bicycle" is returned when a user searches for "bike" or "cycle".
Filtering and sorting: To facilitate browsing and narrowing down searches, our website supports a variety of filtering methods. Users can both sort and filter listings by price, creation and expiration date, whether or not they include photos, and whether they have been marked as favorite. Most of these features were absent in the alternative applications.
Watched searches: Once a user creates the appropriate combination of search query and filters, they can choose to save that combination to come back to it later. Furthermore, they can easily subscribe to get notified whenever a new item matching that search is added.
While given infinite time, we would implement more features, we ultimately decided against many features largely because we determined they weren't worth the time we would need to put into it. (This is made apparent when examining our repository and noting the number of issues we closed because they were low priority.)
- IAS CAS support--We didn't implement this because when we contacted IAS, they explained that we could allow the site, but did not offer test credentials, so we could not justify adding it.
- Textbook-specific features--we knew other groups were already working on this, and decided that the duplicate functionality could be added in later.
- Grid view for viewing more photo-centric listings (e.g. viewing a selection of clothes for sale)--as an improvement over TigerTrade, we figured multiple photo upload was compelling enough as a feature.
- Additional fields for posts:
- Categories, tags
- Location for pickup/dropoff of item
- Renting vs. purchasing prices
Other features we wanted to and thought we had the time to implement, but when meeting with the GSG, as we discuss above, we realized weren't particularly in demand.
- Auctions
- Facebook Messenger integration
- Integration with the Free and For Sale Facebook group
- Suggesting price ranges based on matching items on Amazon
Our team members had wildly varying backgrounds in web development. This meant that we were effectively able to teach each other about different areas of web development. However, in practice, this also often resulted in specialization within areas of prior experience, and occasional holy wars disagreements in methodologies.
Though we all have considerable background in computer science, several team members had little software engineering experience. For those members, the choice of golang for our backend allowed us to see the implementation and structure of a REST API - though this created developer overhead, and though we would not choose to take on that overhead again. By contrast, this understanding would not have been gained using a magical framework like Ruby on Rails. Also, our whole team now has a much stronger understanding of the structure and behavior of a modern single-page web app.
Even the most experienced members of our team still learned important lessons about the challenges of web development. Casey and Perry in particular both came from backgrounds of web development, but Perry came from working in PHP/MySQL in a software development shop, whereas Casey worked more with dynamic languages and their ecosystems, especially Node.js and Rails. When initially discussing databases, Perry worked off the assumption that we would thoroughly specify the database and avoid changing the structure as much as possible, whereas Casey assumed we would start with the minimal schema necessary and iterate. It wasn't until a few weeks in that Casey started mentioning the imminent need to do a "migration", by which he meant modifying the schema, but Perry took this to mean moving the data to another database entirely. Casey had never worked in a development shop, and Perry had never used Rails, where the term migration is common and well-defined.
This kind of disagreement manifested most visibly when we started to decide on design decisions, where our team, being relatively experienced in programming, noticed differences in personal architecture and programming styles, in what is known as bikeshedding. In the backend especially, where no strong conventions existed for architecture, we found ourselves having heated discussions down to whether we should use an ORM or manually write our SQL out.
While this never went away, we got better at realizing when we were bikeshedding and instead deferred to the Internet to find established patterns--some of the patterns are odd, but ultimately prevent us from having to argue over such small details. A place where this shined especially was when using Redux on the frontend. Since there was so much sample code, and because we had enabled an extremely aggresive style checker, we rarely found disagreement when working out implementation details. Conventions exist for a reason.
- Pick the product first and then decide on technology if you don't want to fight your tools. If it feels wrong to use something, it probably is.
- On the other hand, if you want to use a new tool feel free to do so, but don't expect everything to go smoothly.
- Discuss from the beginning how you plan on managing your development. Are you commited to sticking to initial choices of structure and tool sets, or are you willing to go back and change past decisions?
- Good code practices like thorough code reviews may seem spurious in the short term, but save a LOT of time in the long run because you won't be the only one understanding your code.