Add a buffer reader/writer for NTLM.

net/http/ntlm_buffer_writer.h

+// Copyright (c) 2017 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef NET_BASE_NTLM_BUFFER_WRITER_H_
+#define NET_BASE_NTLM_BUFFER_WRITER_H_
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include <memory>
+#include <string>
+
+#include "base/strings/string16.h"
+#include "base/strings/string_piece.h"
+#include "net/base/net_export.h"
+#include "net/http/ntlm.h"
+
+namespace net {
+
+// Supports various bounds checked low level buffer operations required by an
+// NTLM implementation.
+//
+// The class supports sequential write to an internally managed buffer. All
+// writes perform bounds checking to ensure enough space is remaining in the
+// buffer.
+//
+// The internal buffer is allocated in the constructor with size |buffer_len|
+// and owned by the class. Use |ReleaseBufferPtr| to take ownership of the
+// internal buffer. Thereafter all operations on the class will fail.
+//
+//
+// Write* methods write the buffer at the current cursor position and perform
+// any necessary type conversion and provide the data in out params. After a
+// successful write the cursor position is advanced past the written field.
+//
+// Failed writes leave the internal cursor at the same position as before the
+// call.
+//
+//
+// Based on [MS-NLMP]: NT LAN Manager (NTLM) Authentication Protocol
+// Specification version 28.0 [1]
+//
+// [1] https://msdn.microsoft.com/en-us/library/cc236621.aspx
+class NET_EXPORT_PRIVATE NtlmBufferWriter {
+ public:
+  explicit NtlmBufferWriter(size_t buffer_len);
+  ~NtlmBufferWriter();
+
+  size_t GetLength() const { return buffer_len_; }
+  size_t GetCursor() const { return cursor_; }
+  bool IsEndOfBuffer() const { return cursor_ >= GetLength(); }
+
+  // Releases ownership of the internal buffer. Subsequent writes
+  // will all fail.
+  uint8_t* ReleaseBufferPtr() {

[asanka, 2017/06/08 19:40:10]

Why not something like a std::vector<uint8_t> ? That way the caller doesn't need
to know exactly how big the buffer needs to be before the writes.

[zentaro, 2017/06/12 23:12:08]

If the output format was more general/variable like a file encoder I think a
vector which can size as needed would be better. But in this case the format is
small and well defined in size. I feel like it makes it easier to verify if it
is fixed.

Plus at the end of all this the ownership of of a uint8_t* needs to be given to
the auth handler so we'd have to make a copy to a naked buffer anyway.

[asanka, 2017/06/16 03:21:28]

But the caller now needs to know the encoding overheads involved or overshoot
considerably so it doesn't need to worry. Also if the caller needs to specify
the buffer size, it might as well pass in a pointer to the buffer as well. That
way there's no ambiguity around ownership as described below.
I'm not getting why we'd need to make a copy, but I'm also not super concerned
about it since we need to b64 encode it anyway. But it's much more obvious from
an API perspective to transfer a container rather than a raw pointer. Returning
a unique_ptr<uint_8[]> would be an improvement in that it clearly conveys
ownership and hard to get wrong, but a vector would also carry the size with it
which makes consumption much easier.

[zentaro, 2017/06/21 00:38:46]

The caller needs to know the lengths anyway. The first thing written to the
message are the security buffers that write the offset and length of each
payload. So in order to write even the fixed size message header it already
knows the length of the whole message. Having the caller supply the buffer also
seems reasonable.

(I did consider making the writer more complex and initializing with a
'payload_start_index' and maintaining a second cursor into the payload and
adding additional checking that the header section didn't stray into the
payload. But in the end it seemed like added complexity to deal with something
the caller already knows)

Plus in the NTLMv2 case the length of the NTLM response depends on multiple
factors that rely on knowledge way outside the scope of this class that the
caller already has to calculate.

I updated to use a unique_ptr. The copy I was referring to is that
HttpAuthHandlerNTLM::GetNextToken wants to be given ownership of a naked ptr.

+    buffer_len_ = 0;
+    cursor_ = 0;
+    return buffer_.release();
+  }
+
+  // Gets a base::StringPiece view over the entire buffer.
+  base::StringPiece GetBuffer() const {
+    return base::StringPiece(reinterpret_cast<const char*>(buffer_.get()),
+                             buffer_len_);
+  }
+
+  // Returns true if there are |len| more bytes between the current cursor
+  // position and the end of the buffer.
+  bool CanWrite(size_t len) const;
+
+  // Writes a 16 bit unsigned value (little endian). If there are not 16
+  // more bits available in the buffer it returns false.
+  bool WriteUInt16(uint16_t value);
+
+  // Writes a 32 bit unsigned value (little endian). If there are not 32
+  // more bits available in the buffer it returns false.
+  bool WriteUInt32(uint32_t value);
+
+  // Writes a 64 bit unsigned value (little endian). If there are not 64
+  // more bits available in the buffer it returns false.
+  bool WriteUInt64(uint64_t value);
+
+  // Writes |len| bytes from |buffer|. If there are not |len| more bytes in
+  // the buffer it returns false.
+  bool WriteBytes(const uint8_t* buffer, size_t len);
+
+  // Writes the bytes from the |base::StringPiece|. If there are not enough
+  // bytes in the buffer it returns false.
+  bool WriteBytes(base::StringPiece bytes);
+
+  // Writes |count| bytes of zeros to the buffer. If there are not |count|
+  // more bytes in available in the buffer it returns false.
+  bool WriteZeros(size_t count);
+
+  // A security buffer is an 8 byte structure that defines the offset and
+  // length of a payload (string, struct, or byte array) that appears after
+  // the fixed part of the message.
+  //
+  // The structure in the NTLM message is (little endian fields):
+  //     uint16 - |length| Length of payload
+  //     uint16 - Allocation (ignored and always set to |length|)
+  //     uint32 - |offset| Offset from start of message
+  bool WriteSecurityBuffer(ntlm::SecurityBuffer sec_buf);
+
+  // Writes a security buffer with all fields zero. This implies there is
+  // no payload for this field. See |WriteSecurityBuffer|.
+  bool WriteEmptySecurityBuffer();
+
+  // Writes a string of 8 bit characters to the buffer.
+  //
+  // When unicode was not negotiated only the hostname string will go through
+  // this code path. The 8 bit bytes of the hostname are written to the buffer.
+  // The encoding is not relevant.
+  bool WriteNarrowString(const std::string& str);
+
+  // Converts the 16 bit characters to UTF8 and writes the resulting 8 bit
+  // characters.
+  //
+  // If unicode was not negotiated, username and domain get converted to UTF8

[asanka, 2017/06/08 19:40:10]

UTF-8? Shouldn't it need to be CP-1252?

[zentaro, 2017/06/12 23:12:08]

Per other discussion it's actually OEM (basically arbitrary) and for simplicity
of explanation we are just going to consider that our OEM character set is UTF8.
I think most reasonable servers support unicode by now.

+  // in the message. Since they are just treated as additional bytes input to
+  // hash the encoding doesn't matter. In practice only a very old or non-
+  // windows server might trigger this code path since we always attempt to
+  // negotiate unicode and servers are supposed to honor that request.
+  bool WriteString16AsUTF8(const base::string16& str);

[asanka, 2017/06/08 19:40:09]

Nit: prefer to avoid stretches of upper case characters in names. I.e. call it
WriteString16AsUtf8()

[zentaro, 2017/06/12 23:12:08]

Done. Normally I would but I just followed the conventions of all the methods in
base/strings that write UTF all upper case.

+
+  // Treats |str| as UTF8, converts to UTF-16 and writes it with little-endian
+  // byte order to the buffer.
+  //
+  // Two specific strings go through this code path.
+  //
+  // One case is the hostname. When the the unicode flag has been set during
+  // negotiation the hostname needs to appear in the message with 16 bit
+  // chars. The high byte of each char is just set to 0.
+  //
+  // The other case is the spn. With EPA enabled it appears in the target info
+  // inside an AV Pair where strings always have 16 bit chars.
+  bool WriteNarrowStringAsString16(const std::string& str);

[asanka, 2017/06/08 19:40:10]

Nit: Call it WriteUtf8AsUnicodeString since we are calling the previous method
WriteString16AsUtf8 and treating the input as UTF-8.

Also, the next method calls the output format "UnicodeString" which is defined
more narrowly as UTF-16 LE w/o BOM.

[zentaro, 2017/06/12 23:12:07]

Done.

+
+  // Writes UTF-16 LE characters to the buffer. For these strings such as
+  // username and domain the actual encoding isn't really important. They are
+  // just treated as additional bytes of input to the hash.
+  bool WriteUnicodeString(const base::string16& str);
+
+  // Writes the 8 byte NTLM signature "NTLMSSP\0" into the buffer.
+  bool WriteSignature();
+
+  // There are 3 message types Negotiate (sent by client), Challenge (sent by
+  // server), and Authenticate (sent by client).
+  //
+  // This writes |message_type| as a uint32_t into the buffer.
+  bool WriteMessageType(ntlm::MessageType message_type);
+
+  // Performs |WriteSignature| then |WriteMessageType|.
+  bool WriteMessageHeader(ntlm::MessageType message_type);
+
+  // Returns the size in bytes of a string16 depending whether unicode
+  // was negotiated.
+  static size_t GetStringPayloadLength(const base::string16& str,
+                                       bool is_unicode);
+
+  // Returns the size in bytes of a std::string depending whether unicode
+  // was negotiated.
+  static size_t GetStringPayloadLength(const std::string& str, bool is_unicode);

[Ryan Sleevi, 2017/06/08 18:36:34]

Do these need to be public-static?

Could they just be in the unnamed namespace in the .cc? Why or why not? They
feel like an implementation detail :)

[zentaro, 2017/06/21 00:38:46]

Done.

+
+ private:
+  // Writes |sizeof(T)| bytes little-endian of an integer type to the buffer.
+  template <typename T>
+  bool WriteUInt(T value);
+
+  // Sets the cursor position. Will return false if outside the buffer.
+  bool SetCursor(size_t cursor);
+
+  // Returns a pointer to the start of the buffer.
+  uint8_t* GetBufferPtr() const { return buffer_.get(); }
+
+  std::unique_ptr<uint8_t[]> buffer_;
+  size_t buffer_len_;
+  size_t cursor_;
+
+  DISALLOW_COPY_AND_ASSIGN(NtlmBufferWriter);
+};
+
+}  // namespace net
+
+#endif  // NET_BASE_NTLM_BUFFER_WRITER_H_