Subject: support for event-based loops (or workaround for FD_SETSIZE limit)

support for event-based loops (or workaround for FD_SETSIZE limit)

From: Daniel Hardman <daniel.hardman_at_gmail.com>
Date: Thu, 22 Jan 2015 15:13:54 -0700

I need some advice.

I have an app that uses c-ares to issue tens to hundreds of dns requests
per second. It is a daemon that runs forever. I have set the timeout on
these requests very aggressively--but even so, my app is crashing because
sometimes the number of open sockets exceeds 1024, which is greater than
FD_SETSIZE; select() overwrites memory. I am doing all of this with a
single ares channel.

I would love to switch to an epoll-style approach. I found one interaction
between Daniel S and someone else on this forum, where this was discussed.
Daniel said he had added ares_getsock() to support just such a use case.
However, ares_getsock() only supports up to 16 file descriptors, which is
*way* less than what I need. And besides, I don't want to build up a list
of file descriptors each time I iterate through a polling loop; one of the
main benefits of epoll was supposed to be the decoupling of registration of
FDs from monitoring them.

I also saw some posts back in Nov 2013 that floated the idea of supporting
an epoll-like mechanism in cares--but no follow-up announcement of a patch.
There are also some tantalizing hints that the curl team (including
Daniel?) fiddled with c-ares plumbing to support epoll in some manner. But
I can't find any details.

So my question is this: does anybody know of a solid example of event-based
rather than select()- or poll()-based reactor loops for c-ares? If so,
where? If not, do you think I could solve my problem by simply creating
multiple channels, each with a more limited capacity (say, 256 pending
requests)? Or is there a better workaround?

--Daniel Hardman
Received on 2015-01-22