David Cramer's Blog - Latest Comments

Re: 2015 and Beyond - David Cramer's Blog

Tyler Hayes — Wed, 07 Jan 2015 13:50:22 -0000

I look forward to future similar cramer.io announcements. This is awesome.

Re: 2015 and Beyond - David Cramer's Blog

Sherwin Techico — Tue, 06 Jan 2015 06:37:37 -0000

Much luck to you guys! HNY!

Re: 2015 and Beyond - David Cramer's Blog

suki — Mon, 05 Jan 2015 18:02:53 -0000

Congrats, Cramer! And Chris! :)

Re: 2015 and Beyond - David Cramer's Blog

Jeeyoung Kim — Mon, 05 Jan 2015 16:55:59 -0000

This is an awesome news. I've been following Sentry ever since it was release @ Disqus.

Re: 2015 and Beyond - David Cramer's Blog

Thierry Schellenbach — Mon, 05 Jan 2015 11:47:44 -0000

Good luck David, Greetz from GetStream.io

Re: 2015 and Beyond - David Cramer's Blog

Chris Jennings — Mon, 05 Jan 2015 11:44:11 -0000

:clap:

Re: Bundling 3rd party JS with Python - David Cramer's Blog

David Cramer — Mon, 22 Dec 2014 03:04:54 -0000

I actually tried inheriting from build originally and that didn't seem to fire on everything. IIRC the develop command wouldn't fire (but it fired build_py?).

I think realistically the setuptools APIs for builds are just not very well put together.

Re: Bundling 3rd party JS with Python - David Cramer's Blog

Ionel Cristian Mărieș — Sat, 20 Dec 2014 11:58:19 -0000

There's one issue with this: it will make bdist_wheel build incomplete packages. Other commands probably have this issue too.

An alternative is to subclass the `build` command add add your custom `build_static` in the `build.sub_commands` list. Eg:

from setuptools.command.build import build

class BuildWithStatic(build):
    sub_commands = build.sub_commands + [('build_static', lambda _: True)]

With that you don't need to subclass develop, sdist, bdist and whatever else you might need.

Re: Mocking Requests with Responses - David Cramer's Blog

David Shawley — Fri, 23 May 2014 07:31:38 -0000

We went a slightly different way and spin up an HTTPServer instance that records requests and can be preloaded with responses. Having a separate thread running for the test can be error prone. I'll have to investigate switching out our method for responses. I'm also interested in checking out changes. Thanks for putting both of these up on github.

Re: Mocking Requests with Responses - David Cramer's Blog

rosscdh — Wed, 21 May 2014 13:47:27 -0000

Very nice, would be great to be able to record all outgoing requests; and their actual responses (to build the test suite)

Re: Choosing Angular.js - David Cramer's Blog

Alexandru Bardas — Sun, 18 May 2014 19:19:51 -0000

I'm glad that sentry is rewritten in angular and I hope to see updates more often. It's a great tool which helped me & my team a lot.
I've also written about an angular rewrite which dramatically increased my team's productivity ( https://medium.com/web-dev/... ). Angular is powerful at catching errors and was thinking to write about how we integrated angular's $exceptionHandler with raven.js (given that the stack trace in minified js files can often be useless, but user events are not). I'd also like to hear how others did it.

Re: Choosing Angular.js - David Cramer's Blog

David Cramer — Thu, 15 May 2014 12:39:13 -0000

We didn't have any existing Angular code, which is important. Ember had a lot of prescribed things and it was pretty rough to break out of that loop.

Outside of that, the dirty checking in reality isn't as bad as it might seem.

Re: Choosing Angular.js - David Cramer's Blog

Wilson Mendes — Thu, 15 May 2014 12:38:42 -0000

Thanks for share your experience using AngularJs Framework and why was the better choice. I'm using it in my applications and the results are awesome. Very modular, testable and scalable. Talking about dirty-checks exists a decision for update for javascript Object.observe in new version and update directives for webcomponents. Have a presentation about the future of angularjs (https://docs.google.com/pre..., but your can minimize this using modules like bindonce (but is not a better than by default in framework).

Re: Choosing Angular.js - David Cramer's Blog

Dustin Farris — Thu, 15 May 2014 10:08:45 -0000

Thanks for sharing your thoughts on this. It seems your choice of Angular over Ember was due to time constraints and existing code that needed to be integrated. Were there any other technical reasons?

Re: Choosing Angular.js - David Cramer's Blog

Mat Mullen — Thu, 15 May 2014 01:31:47 -0000

"I have a fairly strong opinion on, well, almost everything." - Belongs on a business card or more fitting...a tattoo on your lower back.

Re: Scaling SQL with Redis - David Cramer's Blog

Redsmin {Redis GUI} — Tue, 13 May 2014 08:46:06 -0000

Nice article ! It will be featured in our next @RedisWeekly http://redisweekly.com !

Re: Scaling SQL with Redis - David Cramer's Blog

Dan Parnham — Tue, 13 May 2014 07:48:15 -0000

Rather than use Redis as a lock, we instead use the atomic list operation RPOPLPUSH to allow clients to claim tasks: http://www.emergent-design....

Re: Scaling SQL with Redis - David Cramer's Blog

Michael Dwan — Tue, 13 May 2014 03:43:11 -0000

I've worked around that by verifying a job still retains a lock before writing data to the DB. That worked well when jobs took ~5-10 seconds of idempotent CPU bound work (processing photos and video) before a few quick writes to the db. Lua would certainly be a better option for shorter locks though.

Re: Scaling SQL with Redis - David Cramer's Blog

Giridhar Kannan — Tue, 13 May 2014 03:19:09 -0000

If you are looking for serious lock system then choosing Redis itself is wrong option . May be you can use WAIT command and achieve it, which implies you are choosing consistency over availability.

Re: Scaling SQL with Redis - David Cramer's Blog

David Cramer — Tue, 13 May 2014 02:53:49 -0000

It's definitely a good point. We always faced a big issue with Redis in that a lot of the nice features aren't available with some of the major distros (e.g. 12.04). There's a lot of places we could optimize if we were injecting the Lua scripts (always doing this incr+expire pattern for one).

Re: Scaling SQL with Redis - David Cramer's Blog

churnnews — Tue, 13 May 2014 02:51:37 -0000

Real nice. You might enjoy my hot-redis library which provides a bunch of Python data types backed by Redis, from basic lists and maps (obvious) to different types of locks and queues touched on briefly here.

https://github.com/stephenm...

Re: Scaling SQL with Redis - David Cramer's Blog

David Cramer — Tue, 13 May 2014 02:29:50 -0000

Definitely. Our use case (I mentioned it above) was to ease stampedes on jobs.

For example, we have a cron worker that will fire off jobs, and as a simple hack around distributing the state of that worker we can just fire off jobs from several nodes and attempt to prevent a task from executing at the same time (to prevent wasted effort). In this case you very easily up the expire to a reasonable maximum expected duration of your task.

That said, you could reasonably treat the lock as a counter, and only discard it when the value is 0 or empty.

Re: Scaling SQL with Redis - David Cramer's Blog

Ryan Smith — Tue, 13 May 2014 02:27:51 -0000

FWIW, I have attempted to use Redis as a locking system before and I ended up using Lua to work around the lack of primitives in Redis. As a matter of fact, I am pretty sure the Redis documentation points out the pitfalls of using SETNX as a locking mechanism.

Here is what I ended up doing:
https://github.com/ryandots...

Re: Scaling SQL with Redis - David Cramer's Blog

Ryan Smith — Tue, 13 May 2014 02:26:07 -0000

Are you OK with the race conditions in your locking algorithm? Please correct me if I am wrong, but what about a situation like this:

x acquires lock at t0
x works job at t1
x's lock expires at t9
y acquires lock at t10
x finishes job at t11
x deletes lock at t12
z acquires lock at t13
y & z are both working at t14

Re: On Pull Requests - David Cramer's Blog

David Cramer — Sun, 11 May 2014 01:05:08 -0000

Feature flag always