Subject: Re: curl with c-ares

Re: curl with c-ares

From: William Ahern <william_at_25thandclement.com>
Date: Wed, 4 Sep 2013 12:39:13 -0700

On Wed, Sep 04, 2013 at 02:06:26PM +0200, Daniel Stenberg wrote:
> On Tue, 3 Sep 2013, Shao, Shuchao wrote:
>
> >Via checking the C-ares souce code, there are two large 'for' loops in
> >function ares_init_options(),
>
> >#define ARES_QID_TABLE_SIZE 2048
> >#define ARES_TIMEOUT_TABLE_SIZE 1024
>
> My immediate gut reactions:
>
> 1 - try with much smaller values, like 89 and 47 (primes) and I doubt you'll
> see much of a run-time speed penalty but you'll get much faster initing.
>
> 2 - maybe we should allow these values to get settable with an API? They
> seem to be set to cater for a very large volume of DNS request over a
> single handle and maybe not all users need that. It is also a memory waste
> for the low volume users.
>
> I'm probably in favour of lowering those default values awaiting 2 (unless
> someone reports a major speed penalty).

What I did for dns.c was to write a simple 16-bit feistal block cipher. The
block cipher will permute all possible 16-bit values. So I simply encrypt a
16-bit counter for each request and I'm guaranteed it won't repeat.

A feistal cipher requires a strong cryptographic function at the core, but
it doesn't have to be reversible. I used XTEA, but today the better choice
might be SipHash, although it's not like a 16-bit nonce is super secure to
begin with.

In the long run a block cipher is much slower than using a Fisher-Yates
shuffle, but for most cases the app is only doing a handful of queries, so
in the real-world it executes fewer operations as well as use a fraction of
the memory. And if somebody is doing a ton of queries, it's not like the
cipher even shows up as a blip on the CPU radar. With the cipher approach
the CPU load is smoothed out over the course of the entire QID series rather
than spiking at initialization.

It's similar to the way some DNS systems used LFSRs for QIDs, before
security engineers pointed out how easy it was to predict their output.
Received on 2013-09-04