Adds Wireshark pipes support, cleanup OSX tcpdump handling.

[micolous]

New:

• WiresharkSink, a PipeTools Sink for streaming packets to Wireshark
• utils.get_temp_dir: creates a temporary directory

Extensions:

• tcpdump(): Adds use_tempfile, to explicitly control use of a tempfile for packets.
• tcpdump(): Adds read_stdin_opts parameter, which allows callers to control the options used for reading packets over stdin (previously hard-coded to -r -). This auto-detects wireshark to use -ki - instead.
• tcpdump(), WrpcapSink: Adds linktype parameter.
• get_temp_file(): Adds fd parameter, which allows a temporary file to be used without closing and re-opening it.
• Expands wireshark(), Source and Pipe documentation.
• tdecode(): Add args parameter (defaults to -V, as before), pass kwargs to tcpdump().
• QueueSink.recv: add block and timeout parameters.

Fixes:

• tcpdump() now only uses a temporary file by default for calling tcpdump on OSX (to work around Apple's broken version of tcpdump). stdin is now used with other tools which are not impacted by this bug (eg: tshark).
• wireshark() now uses a pipe rather than a temporary file. Wireshark itself has options to save this file to disk if desired.
• SniffSource now only opens a socket on calling start.
• WrpcapSink only opens a PcapWriter on calling start.
• RawPcapWriter no longer writes a header on write calls with no packets specified.
• RawPcapWriter writes a header on close if no header has been written.
• QueueSink.recv() no longer busy-loops.
• PcapWriter, wrpcap(): support packets as bytes
• PEP-8 fixups and docstring revisions to touched methods.

Documentation fixes:

• Documentation wordsmithing for wireshark() and Pipes
• Use py:function / py:class for Sinks and wireshark() to allow easy cross-referencing, and expand this documentation.
• wireshark(): Remove references to google.com, and use RFC 5737 IP addresses instead
• wireshark(): Set a source IP address (to avoid looking up host's IP address).
• wireshark(): Remove Ether layer, as Wireshark works fine with DLT_RAW for IP packets, which removes the need to lookup MAC addresses
• Add manpages_url to configuration, to support :manpage: directives. This is pointing at Debian's server, which should have a good spread of versions of tools, and Debian generally adds manpages for tools that don't have them.

Test fixes:

• Removes os.remove("test.png") (which seems to be unused).
• Uses a temporary directory for some pipetool tests.

Other changes:

• {Raw,}PcapWriter._write_packet: Remove unused support for packet as tuple, as write will always unroll iterators for us (and do it better).
• {Raw,}PcapWriter._write_packet: Always set the usec parameter if sec was unset.
• {Raw,}PcapWriter._write_packet: Set usec=0 if sec was set, but usec was unset (instead of using the current time's usec value

[gpotter2]

Really like this PR !

If you could rebase, it will fix the AppVeyor failure

[codecov]

Codecov Report

Merging #1959 into master will increase coverage by 0.04%.
The diff coverage is 95.91%.

@@            Coverage Diff             @@
##           master    #1959      +/-   ##
==========================================
+ Coverage   85.92%   85.96%   +0.04%     
==========================================
  Files         187      187              
  Lines       42937    42976      +39     
==========================================
+ Hits        36892    36946      +54     
+ Misses       6045     6030      -15

Impacted Files	Coverage Δ
scapy/pipetool.py	89.91% <100%> (+0.39%)	⬆️
scapy/scapypipes.py	86.71% <100%> (+1.37%)	⬆️
scapy/utils.py	79.15% <93.33%> (+3.05%)	⬆️
scapy/supersocket.py	72.09% <0%> (-2.33%)	⬇️
scapy/layers/tls/handshake_sslv2.py	91.32% <0%> (-2.27%)	⬇️
scapy/layers/tls/record.py	91.22% <0%> (-1.17%)	⬇️
scapy/asn1/ber.py	82.81% <0%> (-0.29%)	⬇️
scapy/layers/ntp.py	91.51% <0%> (-0.27%)	⬇️
scapy/sendrecv.py	85.1% <0%> (-0.17%)	⬇️
scapy/packet.py	79.07% <0%> (-0.17%)	⬇️
... and 3 more

[gpotter2]

Great PR, all good to me apart:

• https://ci.appveyor.com/project/secdev/scapy/builds/23671224/job/cpf5sdranaeooj9k seem to be a random failure of the WiresharkSink test. Probably some racing problems :/ it should be stabilized
• (optional): add a tiny reference in the pipetool documentation (advanced usage) to the new Pipes

[micolous]

Found another issue, tcpdump, wireshark and tshark doesn't like integer linktypes... :(

https://github.com/the-tcpdump-group/tcpdump/blob/c36abfe8a86fb2e3d8256fa53d6569a4aa65b39c/tcpdump.c#L1823-L1829

https://github.com/the-tcpdump-group/libpcap/blob/08dc0344b20dfd44958168d75c3a94ca960eb029/pcap.c#L3109-L3119

[micolous]

In case it isn't clear already, please don't merge this yet...

Yesterday, I was trying to debug that Windows test flake. I ran the test about 1300 times (automated, of course) in Python 3.7 on Linux, and couldn't trigger a single failure.

Interestingly only the winpcap=true version fails.

[micolous]

I'm going to amend the docs a little more, squash and rebase this before it's ready.

[micolous]

@gpotter2: I squashed fixups into commits that make a bit more sense, and fixed the issues you mentioned.

I've updated the original change description with the extra changes since approval, please take a look!

[gpotter2]

the winpcap=True version already fails on some specific tests. It's using a very old libpcap version

About linktypes, we could hack our way through with something similar to

>>> def hacky_linktype_name(value): 
...:     return next(k for k, v in six.iteritems(scapy.data.__dict__) if k[:3] == "DLT" and v == value)
>>>
>>> hacky_linktype_name(0)                              'DLT_NULL'

as a fallback, though that's a bit messy, it could be useful... what do you think ?

Otherwise, the doc improvement is great and PR looks good overall