Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions _config.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
url: http://artsy.github.io
url: https://artsy.github.io
title: Artsy Engineering
subtitle: Discover fine engineering.
google_custom_search: 010245880960712892902:tnjd4ryb3ci
Expand Down Expand Up @@ -174,7 +174,7 @@ authors:

artsy_engineering:
name: The Artsy Engineering Team
site: http://artsy.github.io
site: https://artsy.github.io
twitter: ArtsyOpenSource

nik:
Expand Down
2 changes: 1 addition & 1 deletion _includes/_series_bio/javascriptures.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ frontend tech stack.
We called this JavaScriptures, 6 workshop-style deep dives into the major parts of the [Artsy JavaScript
omakase][omakase]. You can see the YouTube playlist of [them all here][yt].

[omakase]: http://artsy.github.io/blog/2017/02/05/Front-end-JavaScript-at-Artsy-2017/
[omakase]: https://artsy.github.io/blog/2017/02/05/Front-end-JavaScript-at-Artsy-2017/
[yt]: https://www.youtube.com/playlist?list=PLKqXGUTH_9hrr5z97OnL31bmQDDnn2WlU

<hr />
2 changes: 1 addition & 1 deletion _includes/_series_bio/react-native-at-artsy.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ This series covers the reasons why we made that choice, our interactions with Re
and shows how we've been handling some of the issues that come up as an early adopter.

Sidenote: we ran a conference with Facebook in 2018 called
[Artsy x React Native](http://artsy.github.io/artsy-x-react-native.html).
[Artsy x React Native](https://artsy.github.io/artsy-x-react-native.html).
2 changes: 1 addition & 1 deletion _includes/head.html
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
<meta name="MobileOptimized" content="320">
<meta name="viewport" content="width=device-width, initial-scale=1">

<meta property="og:image" content="http://artsy.github.io/images/artsy_oss_image.png" />
<meta property="og:image" content="https://artsy.github.io/images/artsy_oss_image.png" />
<meta property="og:image:type" content="image/png" />

{% capture canonical %}{{ site.url }}{% if site.permalink contains '.html' %}{{ page.url }}{% else %}{{ page.url | remove:'index.html' | strip_slash }}{% endif %}{% endcapture %}
Expand Down
8 changes: 4 additions & 4 deletions _layouts/epic.html
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
</ul>

<ul>
<li><a href="http://artsy.github.io/open-source/">Our Open Source</a></li>
<li><a href="https://artsy.github.io/open-source/">Our Open Source</a></li>
<li><a href="http://twitter.com/artsyopensource">@artsyopensource</a></li>
<li><a href="https://www.artsy.net">artsy.net</a></li>
</ul>
Expand Down Expand Up @@ -74,16 +74,16 @@ <h4>Artsy OSS</h4>
<ul>
<li><a href='https://www.artsy.net'>Artsy.net</a></li>
<li><a href='https://developers.artsy.net'>API</a></li>
<li><a href='http://artsy.github.io/open-source/'>Featured OSS</a></li>
<li><a href='https://artsy.github.io/open-source/'>Featured OSS</a></li>
<li><a href='https://www.artsy.net/jobs'>Careers</a></li>
</ul>
</section>

<section>
<h4>Blog</h4>
<ul>
<li><a href='http://artsy.github.io/blog/archives/'>Archives</a></li>
<li><a href='http://artsy.github.io/search/'>Search</a></li>
<li><a href='https://artsy.github.io/blog/archives/'>Archives</a></li>
<li><a href='https://artsy.github.io/search/'>Search</a></li>
<li><a href='https://github.com/artsy/artsy.github.io'>Code on GitHub</a></li>
<li><a href='https://github.com/artsy/artsy.github.io/edit/source/{{page.path}}'>Fix typoes in this post</a></li>
</ul>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,4 @@ class Person
end
```

With the [mongoid-cached-json](http://github.com/dblock/mongoid-cached-json) gem you also get caching that respects JSON versioning, for free. Read about it [here](http://artsy.github.com/blog/2012/02/20/caching-model-json-with-mongoid-cached-json/).
With the [mongoid-cached-json](http://github.com/dblock/mongoid-cached-json) gem you also get caching that respects JSON versioning, for free. Read about it [here](https://artsy.github.com/blog/2012/02/20/caching-model-json-with-mongoid-cached-json/).
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ bundle install
bundle exec rake
```

Our default Ruby project Rake task is `test:ci`. We run Jasmine and Capybara tests using a real browser, so we need to redirect all visible output to an X-Windows Virtual Frame Buffer ([XVFB](http://www.xfree86.org/4.0.1/Xvfb.1.html)). This can be done by setting an `ENV` variable inside a Rake task. Our test target also [organizes our tests in suites](http://artsy.github.com/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/).
Our default Ruby project Rake task is `test:ci`. We run Jasmine and Capybara tests using a real browser, so we need to redirect all visible output to an X-Windows Virtual Frame Buffer ([XVFB](http://www.xfree86.org/4.0.1/Xvfb.1.html)). This can be done by setting an `ENV` variable inside a Rake task. Our test target also [organizes our tests in suites](https://artsy.github.com/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/).

``` ruby
namespace :test do
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ categories: [Jenkins, Testing, Continuous Integration, EC2]
author: [joey, frank]
---

The [Artsy](http://artsy.net) team faithfully uses [Jenkins](http://jenkins-ci.org) for continuous integration. [As we've described before](http://artsy.github.com/blog/2012/05/27/using-jenkins-for-ruby-and-ruby-on-rails-teams/), our Jenkins master and 8 slaves run on Linode. This arrangement has at least a few drawbacks:
The [Artsy](http://artsy.net) team faithfully uses [Jenkins](http://jenkins-ci.org) for continuous integration. [As we've described before](https://artsy.github.com/blog/2012/05/27/using-jenkins-for-ruby-and-ruby-on-rails-teams/), our Jenkins master and 8 slaves run on Linode. This arrangement has at least a few drawbacks:

* Our Linode servers are manually configured. They require frequent maintenance, and inconsistencies lead to surprising build failures.
* The fixed set of slaves don't match the pattern of our build jobs: jobs get backed up during the day, but servers are mostly unused overnight and on weekends.
Expand Down
2 changes: 1 addition & 1 deletion _posts/2012-10-10-artsy-technology-stack.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ The Artsy CMS and the Admin system are two newer projects and serve the needs of
Folio Partner App
-----------------

Artsy makes a free iOS application, called [Folio](http://artsy.github.com/blog/categories/ios/), which lets our partners display their inventory at art fairs.
Artsy makes a free iOS application, called [Folio](https://artsy.github.com/blog/categories/ios/), which lets our partners display their inventory at art fairs.

Folio is a native iOS implementation. The interface is heavily skinned UIKit with CoreData for storage. Our network code was originally a thin layer on top of NSURLConnection, but for our forthcoming update, we’ve rewritten it to use [AFNetworking](https://github.com/AFNetworking/AFNetworking/). We manage external dependencies with [CocoaPods](https://github.com/CocoaPods/CocoaPods).

Expand Down
4 changes: 2 additions & 2 deletions _posts/2013-04-01-an-easter-egg-for-curl.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ curl https://api.artsy.net/api/v1/system/up?access_token=10013 -v
< Content-Type: application/json
< Content-Length: 76

{ "error" : "Inspiration from the Engineering team at http://artsy.github.com" }
{ "error" : "Inspiration from the Engineering team at https://artsy.github.com" }
```

What?! **401 Broadway**? See, our office address is *401 Broadway, 10013, New York, NY*. We just tried to add a more developer-friendly way to find us in the New York grid. And here's the view from our 25th floor office - that's SOHO right below us and the Empire State Building a bit North.
Expand Down Expand Up @@ -65,7 +65,7 @@ class ApiAuthMiddleware < Grape::Middleware::Base
def before
if access_token == "10013"
throw :error,
message: 'Inspiration from the Engineering team at http://artsy.github.com',
message: 'Inspiration from the Engineering team at https://artsy.github.com',
status: 2600
else
...
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Write fast, headless, tests for Backbone using Node.js. See this project as an e

## A Brief History

Artsy is mostly a thick client [Backbone](http://backbonejs.org/) app that sits on [Rails](http://rubyonrails.org/) and largely depends on [Capybara](http://jnicklas.github.io/capybara/) ([Selenium](http://docs.seleniumhq.org/) backed bot that clicks around Firefox) for testing it's javascript. This leads to some seriously brittle and slow integration tests. [Despite being able to wrangle Capybara](http://artsy.github.io/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/) to do most of our client-side testing, we knew there must be a better way.
Artsy is mostly a thick client [Backbone](http://backbonejs.org/) app that sits on [Rails](http://rubyonrails.org/) and largely depends on [Capybara](http://jnicklas.github.io/capybara/) ([Selenium](http://docs.seleniumhq.org/) backed bot that clicks around Firefox) for testing it's javascript. This leads to some seriously brittle and slow integration tests. [Despite being able to wrangle Capybara](https://artsy.github.io/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/) to do most of our client-side testing, we knew there must be a better way.

When building a CMS app for our gallery partners to manage their Artsy inventory, we built a new Backbone app on top of [node.js](http://nodejs.org/). The result was a headless test suite that runs around 60 times faster.

Expand Down
2 changes: 1 addition & 1 deletion _posts/2013-08-27-introduction-to-aws-opsworks.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Conversely, OpsWorks offers higher-level control than [CloudFormation](https://a

{% include expanded_img.html url="/images/2013-08-27-introduction-to-aws-opsworks/aws_control.png" title="How OpsWorks fits in AWS offerings" %}

Historically, Artsy delegated dev-ops concerns to Heroku. They worried about infrastructure, freeing us to focus on our application's higher-level goals. Increasingly though, we were forced to work around limitations of the platform's performance, architecture, and customizability. (We even blogged about it [here](http://artsy.github.io/blog/2012/01/31/beyond-heroku-satellite-delayed-job-workers-on-ec2/), [here](http://artsy.github.io/blog/2012/11/15/how-to-monitor-503s-and-timeout-on-heroku/), [here](http://artsy.github.io/blog/2012/12/13/beat-heroku-60-seconds-application-boot-timeout-with-a-proxy/), [here](http://artsy.github.io/blog/2013/02/01/master-heroku-command-line-with-heroku-commander/), and [here](http://artsy.github.io/blog/2013/02/17/impact-of-heroku-routing-mesh-and-random-routing/).) Rather than continue to work against the platform, we turned to OpsWorks for greater flexibility while keeping administrative burden low.
Historically, Artsy delegated dev-ops concerns to Heroku. They worried about infrastructure, freeing us to focus on our application's higher-level goals. Increasingly though, we were forced to work around limitations of the platform's performance, architecture, and customizability. (We even blogged about it [here](https://artsy.github.io/blog/2012/01/31/beyond-heroku-satellite-delayed-job-workers-on-ec2/), [here](https://artsy.github.io/blog/2012/11/15/how-to-monitor-503s-and-timeout-on-heroku/), [here](https://artsy.github.io/blog/2012/12/13/beat-heroku-60-seconds-application-boot-timeout-with-a-proxy/), [here](https://artsy.github.io/blog/2013/02/01/master-heroku-command-line-with-heroku-commander/), and [here](https://artsy.github.io/blog/2013/02/17/impact-of-heroku-routing-mesh-and-random-routing/).) Rather than continue to work against the platform, we turned to OpsWorks for greater flexibility while keeping administrative burden low.


## OpsWorks Overview
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ These failures might be due to slight timing differences or lack of proper isola

## The Quarantine

We've been [automatically retrying failed tests](http://artsy.github.io/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/), with some success. However, these problems tend to get worse. (If you have 10 tests that each have a 1% chance of failing, roughly 1 in 10 builds will fail. If you have 50, 4 in 10 builds will fail.)
We've been [automatically retrying failed tests](https://artsy.github.io/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/), with some success. However, these problems tend to get worse. (If you have 10 tests that each have a 1% chance of failing, roughly 1 in 10 builds will fail. If you have 50, 4 in 10 builds will fail.)

Martin Fowler offers the most compelling thoughts on this topic in [Eradicating Non-Determinism in Tests](http://martinfowler.com/articles/nonDeterminism.html). (Read it, really.) He suggests quarantining problematic tests in a separate suite, so they don't block the build pipeline.

Expand Down
2 changes: 1 addition & 1 deletion _posts/2014-04-02-refactoring-infrastructure.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Once the new app was feature-complete, we removed the proxying layer and updated
New hosting environment
---

When we [switched hosting providers](http://artsy.github.io/blog/2013/08/27/introduction-to-aws-opsworks/) for our main application, it required changes to _many_ related tools and services (for logs, deploys, background queues, etc.). To mitigate risk, we set up a "double-deploy" to the legacy and new environments as soon as the basic elements were in place. The environments ran the same code and shared a data store. First, we targeted the new environment from only a few internal apps. As we surfaced and fixed bugs, we directed more client applications away from the old, eventually winding it down altogether. The process was spread over months, but since each individual change was small and low-risk, we were confident and could adjust course as necessary.
When we [switched hosting providers](https://artsy.github.io/blog/2013/08/27/introduction-to-aws-opsworks/) for our main application, it required changes to _many_ related tools and services (for logs, deploys, background queues, etc.). To mitigate risk, we set up a "double-deploy" to the legacy and new environments as soon as the basic elements were in place. The environments ran the same code and shared a data store. First, we targeted the new environment from only a few internal apps. As we surfaced and fixed bugs, we directed more client applications away from the old, eventually winding it down altogether. The process was spread over months, but since each individual change was small and low-risk, we were confident and could adjust course as necessary.

Full application rewrite
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ categories: [Ruby, Email, CSS, HTML]
author: matt
---

We recently launched a new personalized email here at [Artsy](https://artsy.net) that features content that a given user might find interesting. The goal of this post is to describe how we built a backend system that efficiently generates these e-mails for all our users. I'll talk about the first, naive implementation that had performance problems right away, and how the second implementation (currently in production) solved those issues, and whose behavior at scale is well-defined and understood. I won't go into the details of the design and layout of the mail itself and how we render the content - there are several earlier blog posts that deal with those: [Presenters and Memoization](http://artsy.github.io/blog/2014/03/18/presenters-and-memoization-moving-logic-out-of-templates/), [Pinterest-style Layouts](http://artsy.github.io/blog/2014/03/17/ruby-helper-to-group-artworks-into-a-pinterest-style-layout-for-email/) and [Email Layouts and Responsiveness](http://artsy.github.io/blog/2014/03/17/some-tips-for-email-layout-and-responsiveness/).
We recently launched a new personalized email here at [Artsy](https://artsy.net) that features content that a given user might find interesting. The goal of this post is to describe how we built a backend system that efficiently generates these e-mails for all our users. I'll talk about the first, naive implementation that had performance problems right away, and how the second implementation (currently in production) solved those issues, and whose behavior at scale is well-defined and understood. I won't go into the details of the design and layout of the mail itself and how we render the content - there are several earlier blog posts that deal with those: [Presenters and Memoization](https://artsy.github.io/blog/2014/03/18/presenters-and-memoization-moving-logic-out-of-templates/), [Pinterest-style Layouts](https://artsy.github.io/blog/2014/03/17/ruby-helper-to-group-artworks-into-a-pinterest-style-layout-for-email/) and [Email Layouts and Responsiveness](https://artsy.github.io/blog/2014/03/17/some-tips-for-email-layout-and-responsiveness/).

![Personalized Email Example](/images/2014-04-24-generating-notifications-and-personalized-emails-efficiently/percy_example.png)

Expand Down Expand Up @@ -209,7 +209,7 @@ def notifiable?
end
```

And...that's basically it! Throughout the week as partners are uploading their shows/fair booths/artworks, these records are being opportunistically written to that day's notification collection, in a performant and scalable fashion. Then when we want to send you a personalized email, we pull all your appropriate notifications via the ```get``` routine in our ```NotificationService```, and primarily using the technique described in [Presenters and Memoization](http://artsy.github.io/blog/2014/03/18/presenters-and-memoization-moving-logic-out-of-templates/) we make sure we cache/memoize all such data. Using the tips in [Pinterest-style Layouts](http://artsy.github.io/blog/2014/03/17/ruby-helper-to-group-artworks-into-a-pinterest-style-layout-for-email/) and [Email Layouts and Responsiveness](http://artsy.github.io/blog/2014/03/17/some-tips-for-email-layout-and-responsiveness/) we can render this content and support various devices/email clients. We parallelize and batch the generation/sending of our e-mails as well. This whole system, from notification generation to actually emailing users, is running successfully and smoothly in production.
And...that's basically it! Throughout the week as partners are uploading their shows/fair booths/artworks, these records are being opportunistically written to that day's notification collection, in a performant and scalable fashion. Then when we want to send you a personalized email, we pull all your appropriate notifications via the ```get``` routine in our ```NotificationService```, and primarily using the technique described in [Presenters and Memoization](https://artsy.github.io/blog/2014/03/18/presenters-and-memoization-moving-logic-out-of-templates/) we make sure we cache/memoize all such data. Using the tips in [Pinterest-style Layouts](https://artsy.github.io/blog/2014/03/17/ruby-helper-to-group-artworks-into-a-pinterest-style-layout-for-email/) and [Email Layouts and Responsiveness](https://artsy.github.io/blog/2014/03/17/some-tips-for-email-layout-and-responsiveness/) we can render this content and support various devices/email clients. We parallelize and batch the generation/sending of our e-mails as well. This whole system, from notification generation to actually emailing users, is running successfully and smoothly in production.

### Next Steps

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -276,8 +276,8 @@ This basic structure has accommodated dozens of test scenarios. We've extended i

A caveat: with so many layers and dependencies involved, there are often spurious failures. We've picked up a few practices that help:

* [Automatic retries](http://artsy.github.io/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/)
* [Quarantine for problematic tests](http://artsy.github.io/blog/2014/01/30/isolating-spurious-and-nondeterministic-tests/)
* [Automatic retries](https://artsy.github.io/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/)
* [Quarantine for problematic tests](https://artsy.github.io/blog/2014/01/30/isolating-spurious-and-nondeterministic-tests/)
* [Failure screenshots](https://github.com/mattheworiordan/capybara-screenshot)

You can [grab the example code](https://github.com/joeyAghion/multiapp_example-tests). And make sure to let us know in the comments how _you_ approach testing across applications.
Loading