fix for ubsan on unittest.h fastrand()

fix for ubsan on unittest.h fastrand()

internal math of the fastrand function uses a multiply
and add that overflows a signed int.  This triggers a
ubsan failure:

../../unit_test/../unit_test/unit_test.h:60:33: runtime error: signed integer overflow: 56248274 * 214013 cannot be represented in type 'int'

This change casts the intermediate math to unsigned
int to avoid the overflow.

For more info on ubsan, see
http://dev.chromium.org/developers/testing/undefinedbehaviorsanitizer

TESTED=Passing compilation using:
GYP_DEFINES="ubsan=1"
GYP_DEFINES="ubsan_vptr=1"

R=harryjin@google.com, pbos@webrtc.org
BUG=libyuv:563
Committed: https://chromium.googlesource.com/libyuv/libyuv/+/903c91cc2e3c831748d6f6168b7a8bb98f6ab9ec

[fbarchard1, 2016-02-02 20:24:42]

Description was changed from

==========
cast random function for ubsan fix

rework scale code for ubsan

For more info on ubsan, see
http://dev.chromium.org/developers/testing/undefinedbehaviorsanitizer

TESTED=Passing compilation using:
GYP_DEFINES="ubsan=1"
GYP_DEFINES="ubsan_vptr=1"

R=harryjin@google.com,pbos@webrtc.org
BUG=libyuv:563
==========

to

==========
fix for ubsan on unittest.h fastrand()

internal math of the fastrand function uses a multiply
and add that overflows a signed int.  This triggers a
ubsan failure:

../../unit_test/../unit_test/unit_test.h:60:33: runtime error: signed integer
overflow: 56248274 * 214013 cannot be represented in type 'int'

This change casts the intermediate math to unsigned
int to avoid the overflow.

For more info on ubsan, see
http://dev.chromium.org/developers/testing/undefinedbehaviorsanitizer

TESTED=Passing compilation using:
GYP_DEFINES="ubsan=1"
GYP_DEFINES="ubsan_vptr=1"

R=harryjin@google.com,pbos@webrtc.org
BUG=libyuv:563
==========

[fbarchard1, 2016-02-02 20:28:18]

GYP_DEFINES=ubsan=1 gclient runhooks
ninja -v -C out/Release
out/Release/libyuv_unittest

[pbos-webrtc, 2016-02-02 21:11:47]

https://codereview.chromium.org/1662453003/diff/1/unit_test/unit_test.h
File unit_test/unit_test.h (right):

https://codereview.chromium.org/1662453003/diff/1/unit_test/unit_test.h#newco...
unit_test/unit_test.h:58: extern int fastrand_seed;
Just use an unsigned int, then fastrand_seed >> 16 % 0xFFFFu casts to an int
without overflows.

[harryjin, 2016-02-02 21:39:51]

lgtm

[fbarchard1, 2016-02-02 22:23:33]

added version number

https://codereview.chromium.org/1662453003/diff/1/unit_test/unit_test.h
File unit_test/unit_test.h (right):

https://codereview.chromium.org/1662453003/diff/1/unit_test/unit_test.h#newco...
unit_test/unit_test.h:58: extern int fastrand_seed;
On 2016/02/02 21:11:47, pbos-webrtc wrote:
> Just use an unsigned int, then fastrand_seed >> 16 % 0xFFFFu casts to an int
> without overflows.

Done.

[fbarchard1, 2016-02-02 22:32:15]

Description was changed from

==========
fix for ubsan on unittest.h fastrand()

internal math of the fastrand function uses a multiply
and add that overflows a signed int.  This triggers a
ubsan failure:

../../unit_test/../unit_test/unit_test.h:60:33: runtime error: signed integer
overflow: 56248274 * 214013 cannot be represented in type 'int'

This change casts the intermediate math to unsigned
int to avoid the overflow.

For more info on ubsan, see
http://dev.chromium.org/developers/testing/undefinedbehaviorsanitizer

TESTED=Passing compilation using:
GYP_DEFINES="ubsan=1"
GYP_DEFINES="ubsan_vptr=1"

R=harryjin@google.com,pbos@webrtc.org
BUG=libyuv:563
==========

to

==========
fix for ubsan on unittest.h fastrand()

internal math of the fastrand function uses a multiply
and add that overflows a signed int.  This triggers a
ubsan failure:

../../unit_test/../unit_test/unit_test.h:60:33: runtime error: signed integer
overflow: 56248274 * 214013 cannot be represented in type 'int'

This change casts the intermediate math to unsigned
int to avoid the overflow.

For more info on ubsan, see
http://dev.chromium.org/developers/testing/undefinedbehaviorsanitizer

TESTED=Passing compilation using:
GYP_DEFINES="ubsan=1"
GYP_DEFINES="ubsan_vptr=1"

R=harryjin@google.com, pbos@webrtc.org
BUG=libyuv:563

Committed:
https://chromium.googlesource.com/libyuv/libyuv/+/903c91cc2e3c831748d6f6168b7...
==========

[fbarchard1, 2016-02-02 22:32:16]

Committed patchset #3 (id:40001) manually as
903c91cc2e3c831748d6f6168b7a8bb98f6ab9ec (presubmit successful).