Subject: Re: [Bug-wget] Anyone want to add libcares support to wget?

Re: [Bug-wget] Anyone want to add libcares support to wget?

From: Ben Greear <greearb_at_candelatech.com>
Date: Fri, 18 Mar 2016 08:58:49 -0700

On 03/18/2016 02:33 AM, Tim Ruehsen wrote:
>
> Tim
>
> On Thursday 17 March 2016 09:36:47 Ben Greear wrote:
>> On 03/17/2016 08:58 AM, Giuseppe Scrivano wrote:
>>> Hi Ben,
>>>
>>> Ben Greear <greearb_at_candelatech.com> writes:
>>>>> Did you consider using a container (e.g. docker) for such a task ? Easy
>>>>> to set up and you'll have your feature not only for wget. IMO, that is
>>>>> much more flexible. (It was Giuseppe's idea during a private talk).
>>>>
>>>> Containers will not work for me. I need to scale to thousands of
>>>> instances on modest hardware. I'm certain the libcares and binding
>>>> approach will work because we do similar things with curl and other
>>>> programs already.
>>>
>>> how modest is this hardware?
>>>
>>> I tried to build a minimal Docker container for wget on top of Alpine:
>>>
>>> Dockerfile:
>>>
>>> FROM alpine
>>> RUN apk add --update wget
>>> WORKDIR /out
>>> ENTRYPOINT ["/usr/bin/wget"]
>>>
>>> and simply running (192.168.1.13 is an internal DNS server):
>>>
>>> docker run --dns=192.168.1.13 --rm -v $(pwd):/out:Z wget wdserver
>>>
>>> takes around 5M, plus the docker daemon.
>>>
>>> If this is not enough, you can even just run wget in a chroot, and
>>> provide a different /etc/resolv.conf.
>>
>> I really am not at all interested in LD_PRELOAD, docker, chroot, or any
>> other such a thing. I have 15+ years invested in my current architecture,
>> and it works very well for me. It absolutely cannot be easier to manage
>> docker instances on random platforms vs using a few cmd-line switches to
>> wget.
>>
>> I know that this feature will not be useful for most people, but then again,
>> none of us can know what some user might be able to do with such a feature.
>> I know that some users of curl + libcares wanted the ability to specify
>> certain DNS servers to get around oppressive firewalls (on systems they
>> could not have root privs), for instance.
>>
>> I don't mind making the changes to wget. I thought if someone was
>> interested in a project, then I'd be happy to work with them (and pay them,
>> etc).
>
> Why not put an offer here (with an exact description of what to deliver, OS,
> etc) ?

A lot of lists do not like such postings, so I was trying to be a bit discreet.

Anyway, I'll pay $500 USD for a working implementation, and $1000 if someone
can get it coded up well enough that it is accepted upstream. If someone thinks
this is not an appropriate amount, let me know your suggestions.

Features:

  * Ability to link with libcares for all DNS lookup requests.
  * Ability to specify DNS server list for libcares. At least IPv4. IPv6 is
    nice but not required.
  * Ability to bind libcares to a local IP address. Ability to also bind to local
    network interface is nice but not required. libcares library already supports this,
    see 'curl' for examples of how this was done.
  * All of this configurable with cmd-line arguments.
  * Ability to specify which libcares to include and link against (with ./configure option or similar)

  * OS: Linux, modern Fedora.

>
>
>> I'll post my patches for consideration, and you will be welcome to pull it
>> into your tree. If they are not acceptable, then I can maintain the fork
>> for as long as my customers care to use it..
>
> Your patches are welcome, no doubt.
>
> Just please understand our position that we try to find solutions by other
> means before introducing new options.

I understand, and I don't mind the suggestions...its just that in this case,
I am pretty certain what I need and want, and some of the more generic work-arounds
suggested just do not meet my needs.

Thanks,
Ben

>
> Regards, Tim
>

-- 
Ben Greear <greearb_at_candelatech.com>
Candela Technologies Inc  http://www.candelatech.com
Received on 2016-03-18