|
|
DescriptionDo not load main cross ref table for first page in linearized pdf.
For the first page in linearized PDFs, it is not necessary to load the
main xref table, because it have separate special xref table in the file
header.
This decreasing the time to show first page in PDF plugin.
Committed: https://pdfium.googlesource.com/pdfium/+/61f8e9c5aeb0d8cb5477e0248b685214746bada7
Patch Set 1 #Patch Set 2 : Do not load main cross refs for first page in linearized pdf. #Patch Set 3 : Fix XFA pages availability.` #
Total comments: 9
Patch Set 4 : Fix review issues.` #Patch Set 5 : Fix review issues.` #Patch Set 6 : fix compilation. #Patch Set 7 : fix compilation. #
Messages
Total messages: 61 (34 generated)
Description was changed from ========== Do not load main cross refs for first page in linearized pdf. For fist page in linearized pdf is not necessary to load main xref table. This decreasing the time to show first page in PDF plugin. ========== to ========== Do not load main cross ref table for first page in linearized pdf. For fist page in linearized pdf is not necessary to load main xref table, because it have separate special xref table at file begin. This decreasing the time to show first page in PDF plugin. ==========
art-snake@yandex-team.ru changed reviewers: + brucedawson@chromium.org, dsinclair@chromium.org, thestig@chromium.org
The CQ bit was checked by art-snake@yandex-team.ru to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: No L-G-T-M from a valid reviewer yet. CQ run can only be started by full committers or once the patch has received an L-G-T-M from a full committer. Even if an L-G-T-M may have been provided, it was from a non-committer, _not_ a full super star committer. Committers are members of the group "project-pdfium-committers". Note that this has nothing to do with OWNERS files.
The CQ bit was checked by art-snake@yandex-team.ru to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: No L-G-T-M from a valid reviewer yet. CQ run can only be started by full committers or once the patch has received an L-G-T-M from a full committer. Even if an L-G-T-M may have been provided, it was from a non-committer, _not_ a full super star committer. Committers are members of the group "project-pdfium-committers". Note that this has nothing to do with OWNERS files.
Deferring to thestig@ for linearized changes.
The CQ bit was checked by thestig@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: win_xfa_clang on master.tryserver.client.pdfium (JOB_FAILED, https://build.chromium.org/p/tryserver.client.pdfium/builders/win_xfa_clang/b...)
The trybots are red. Please take a look. Also, I can't remember under what exact conditions ASSERT() calls run. Since the ValidateFoo() methods are not const, their may be a subtle difference when ASSERTs are enabled vs. disabled.
On 2016/11/08 10:53:02, Lei Zhang (slow) wrote: > The trybots are red. Please take a look. > > Also, I can't remember under what exact conditions ASSERT() calls run. Since the > ValidateFoo() methods are not const, their may be a subtle difference when > ASSERTs are enabled vs. disabled. When ASSERTs are disabled, do nothing: https://cs.chromium.org/chromium/src/third_party/pdfium/core/fxcrt/fx_system....
On 2016/11/08 10:53:02, Lei Zhang (slow) wrote: > The trybots are red. Please take a look. > > Also, I can't remember under what exact conditions ASSERT() calls run. Since the > ValidateFoo() methods are not const, their may be a subtle difference when > ASSERTs are enabled vs. disabled. I have researched, that xfa specific test files (on which tests are failed): PagePosition_any_rest.pdf PagePosition_first.pdf PagePosition_last.pdf PagePosition_no_any.pdf PagePosition_rest.pdf PagePosition_rest_any.pdf have incorrect linearized format: The linearized header conatins valid file size, but PagesCount and Hint Table is invalid. LinearizedPageCount(always 1) != RealPageCount. Hint table is empty(like we have only first page), but should contains data for other pages, To fix this i just want to break file size too. How i can create CL into https://pdfium.googlesource.com/pdfium_tests ? When i have try to call "git cl upload", I have error.
On 2016/11/08 15:18:46, snake wrote: > On 2016/11/08 10:53:02, Lei Zhang (slow) wrote: > > The trybots are red. Please take a look. > > > > Also, I can't remember under what exact conditions ASSERT() calls run. Since > the > > ValidateFoo() methods are not const, their may be a subtle difference when > > ASSERTs are enabled vs. disabled. > > I have researched, that xfa specific test files (on which tests are failed): > PagePosition_any_rest.pdf > PagePosition_first.pdf > PagePosition_last.pdf > PagePosition_no_any.pdf > PagePosition_rest.pdf > PagePosition_rest_any.pdf > > have incorrect linearized format: > The linearized header conatins valid file size, but PagesCount and Hint Table is > invalid. > LinearizedPageCount(always 1) != RealPageCount. > Hint table is empty(like we have only first page), but should contains data for > other pages, In a non-XFA capable PDF viewer, there is only 1 page - the one that tells the user to go download Adobe Reader. Perhaps the page count of 1 refers to that? > To fix this i just want to break file size too. > How i can create CL into https://pdfium.googlesource.com/pdfium_tests ? > When i have try to call "git cl upload", I have error. If you are in testing/corpus, that is a separate git repo. You should be able follow the same process there as you do in any other repo - start a new branch, commit to the branch, git cl upload. Be sure to sync to HEAD first.
The CQ bit was checked by art-snake@yandex-team.ru to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: No L-G-T-M from a valid reviewer yet. CQ run can only be started by full committers or once the patch has received an L-G-T-M from a full committer. Even if an L-G-T-M may have been provided, it was from a non-committer, _not_ a full super star committer. Committers are members of the group "project-pdfium-committers". Note that this has nothing to do with OWNERS files.
Thank you for information about XFA specific. I have fixed tests. https://codereview.chromium.org/2483633002/diff/40001/core/fpdfapi/parser/cpd... File core/fpdfapi/parser/cpdf_data_avail.cpp (left): https://codereview.chromium.org/2483633002/diff/40001/core/fpdfapi/parser/cpd... core/fpdfapi/parser/cpdf_data_avail.cpp:1546: return DataAvailable; I back this line, because it is related to XFA pages checking.
The CQ bit was checked by thestig@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
On 2016/11/08 12:32:43, snake wrote: > On 2016/11/08 10:53:02, Lei Zhang (slow) wrote: > > The trybots are red. Please take a look. > > > > Also, I can't remember under what exact conditions ASSERT() calls run. Since > the > > ValidateFoo() methods are not const, their may be a subtle difference when > > ASSERTs are enabled vs. disabled. > > When ASSERTs are disabled, do nothing: > https://cs.chromium.org/chromium/src/third_party/pdfium/core/fxcrt/fx_system.... Right, so for example, |m_ObjectSet| and the internal state of |m_pDocument| can be different depending on whether ASSERTs are enabled or not. The code can thus behave differently depending on whether it is debug mode or release mode.
https://codereview.chromium.org/2483633002/diff/40001/core/fpdfapi/parser/cpd... File core/fpdfapi/parser/cpdf_data_avail.cpp (right): https://codereview.chromium.org/2483633002/diff/40001/core/fpdfapi/parser/cpd... core/fpdfapi/parser/cpdf_data_avail.cpp:1352: if (!m_pLinearized || !m_pLinearized->GetLastXRefOffset()) Should be safe to remove the !m_pLinearized check. https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... File fpdfsdk/fpdf_dataavail_embeddertest.cpp (right): https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... fpdfsdk/fpdf_dataavail_embeddertest.cpp:207: const int first_page_num = FPDFAvail_GetFirstPageNum(document_); Might as well check the result since we have it? https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... fpdfsdk/fpdf_dataavail_embeddertest.cpp:214: // No new data available, to took the download request. "to took the download request" doesn't parse. https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... fpdfsdk/fpdf_dataavail_embeddertest.cpp:216: FPDFAvail_IsPageAvail(avail_, first_page_num, loader.hints()); Do you care about the return value?
https://codereview.chromium.org/2483633002/diff/40001/core/fpdfapi/parser/cpd... File core/fpdfapi/parser/cpdf_data_avail.cpp (right): https://codereview.chromium.org/2483633002/diff/40001/core/fpdfapi/parser/cpd... core/fpdfapi/parser/cpdf_data_avail.cpp:1352: if (!m_pLinearized || !m_pLinearized->GetLastXRefOffset()) On 2016/11/09 08:51:11, Lei Zhang (slow) wrote: > Should be safe to remove the !m_pLinearized check. Done. https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... File fpdfsdk/fpdf_dataavail_embeddertest.cpp (right): https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... fpdfsdk/fpdf_dataavail_embeddertest.cpp:207: const int first_page_num = FPDFAvail_GetFirstPageNum(document_); On 2016/11/09 08:51:12, Lei Zhang (slow) wrote: > Might as well check the result since we have it? We always should have it. For this test is not matter, what value of this. https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... fpdfsdk/fpdf_dataavail_embeddertest.cpp:214: // No new data available, to took the download request. On 2016/11/09 08:51:12, Lei Zhang (slow) wrote: > "to took the download request" doesn't parse. Done. https://codereview.chromium.org/2483633002/diff/40001/fpdfsdk/fpdf_dataavail_... fpdfsdk/fpdf_dataavail_embeddertest.cpp:216: FPDFAvail_IsPageAvail(avail_, first_page_num, loader.hints()); On 2016/11/09 08:51:12, Lei Zhang (slow) wrote: > Do you care about the return value? No. For first call this is not matter. (Currently the first page availability is checked as part of document availability check, and on first call it is already available, but this can be changed in future and not related to this test). But on second call it is should be available always (in this test case).
The CQ bit was checked by thestig@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
On 2016/11/09 08:50:55, Lei Zhang (slow) wrote: > On 2016/11/08 12:32:43, snake wrote: > > On 2016/11/08 10:53:02, Lei Zhang (slow) wrote: > > > The trybots are red. Please take a look. > > > > > > Also, I can't remember under what exact conditions ASSERT() calls run. Since > > the > > > ValidateFoo() methods are not const, their may be a subtle difference when > > > ASSERTs are enabled vs. disabled. > > > > When ASSERTs are disabled, do nothing: > > > https://cs.chromium.org/chromium/src/third_party/pdfium/core/fxcrt/fx_system.... > > Right, so for example, |m_ObjectSet| and the internal state of |m_pDocument| can > be different depending on whether ASSERTs are enabled or not. The code can thus > behave differently depending on whether it is debug mode or release mode. We should still discuss ASSERTs that have side effects. One possible solution is to always run the validation. e.g. bool ret = ValidateFoo(); ASSERT(ret);
On 2016/11/09 21:53:29, Lei Zhang (slow) wrote: > On 2016/11/09 08:50:55, Lei Zhang (slow) wrote: > > On 2016/11/08 12:32:43, snake wrote: > > > On 2016/11/08 10:53:02, Lei Zhang (slow) wrote: > > > > The trybots are red. Please take a look. > > > > > > > > Also, I can't remember under what exact conditions ASSERT() calls run. > Since > > > the > > > > ValidateFoo() methods are not const, their may be a subtle difference when > > > > ASSERTs are enabled vs. disabled. > > > > > > When ASSERTs are disabled, do nothing: > > > > > > https://cs.chromium.org/chromium/src/third_party/pdfium/core/fxcrt/fx_system.... > > > > Right, so for example, |m_ObjectSet| and the internal state of |m_pDocument| > can > > be different depending on whether ASSERTs are enabled or not. The code can > thus > > behave differently depending on whether it is debug mode or release mode. > > We should still discuss ASSERTs that have side effects. One possible solution is > to always run the validation. e.g. > > bool ret = ValidateFoo(); > ASSERT(ret); Only as temporary solution. Are you agree?
On 2016/11/09 23:05:45, snake wrote: > On 2016/11/09 21:53:29, Lei Zhang (slow) wrote: > > We should still discuss ASSERTs that have side effects. One possible solution > is > > to always run the validation. e.g. > > > > bool ret = ValidateFoo(); > > ASSERT(ret); > > Only as temporary solution. Are you agree? Do you have any thoughts on a permanent solution? What if we drop the validation methods altogether?
On 2016/11/09 23:10:20, Lei Zhang (slow) wrote: > On 2016/11/09 23:05:45, snake wrote: > > On 2016/11/09 21:53:29, Lei Zhang (slow) wrote: > > > We should still discuss ASSERTs that have side effects. One possible > solution > > is > > > to always run the validation. e.g. > > > > > > bool ret = ValidateFoo(); > > > ASSERT(ret); > > > > Only as temporary solution. Are you agree? > > Do you have any thoughts on a permanent solution? > > What if we drop the validation methods altogether? Ww should refactore the AreObjectsAvailable to make it const, and all members of CPDF_Aavail transfer to it. (or dummy variabled for ValideteXXX methods) I think we should not drop them. No other checks or tests for this now (I have found much invalid cases, uses it, like in neighboring CL with page without ObjNum).
Description was changed from ========== Do not load main cross ref table for first page in linearized pdf. For fist page in linearized pdf is not necessary to load main xref table, because it have separate special xref table at file begin. This decreasing the time to show first page in PDF plugin. ========== to ========== Do not load main cross ref table for first page in linearized pdf. For the first page in linearized PDFs, it is not necessary to load the main xref table, because it have separate special xref table in the file header. This decreasing the time to show first page in PDF plugin. ==========
On 2016/11/09 23:19:11, snake wrote: > Ww should refactore the AreObjectsAvailable to make it const, and all members of > CPDF_Aavail transfer to it. (or dummy variabled for ValideteXXX methods) > > I think we should not drop them. > No other checks or tests for this now (I have found much invalid cases, uses it, > like in neighboring CL with page without ObjNum). Ok sounds good. Can we go with the bool ret = Validate() and ASSERT(ret) solution for now? I rewrote the CL description slightly.
On 2016/11/09 23:30:00, Lei Zhang (slow) wrote: > On 2016/11/09 23:19:11, snake wrote: > > Ww should refactore the AreObjectsAvailable to make it const, and all members > of > > CPDF_Aavail transfer to it. (or dummy variabled for ValideteXXX methods) > > > > I think we should not drop them. > > No other checks or tests for this now (I have found much invalid cases, uses > it, > > like in neighboring CL with page without ObjNum). > > Ok sounds good. Can we go with the bool ret = Validate() and ASSERT(ret) > solution for now? > > I rewrote the CL description slightly. Done
LGTM, thanks.
The CQ bit was checked by art-snake@yandex-team.ru
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: mac_xfa_rel on master.tryserver.client.pdfium (JOB_FAILED, https://build.chromium.org/p/tryserver.client.pdfium/builders/mac_xfa_rel/bui...)
On 2016/11/10 00:02:15, commit-bot: I haz the power wrote: > Try jobs failed on following builders: > mac_xfa_rel on master.tryserver.client.pdfium (JOB_FAILED, > https://build.chromium.org/p/tryserver.client.pdfium/builders/mac_xfa_rel/bui...) Can i add (void)&is_page_valid; to fix "unused variable" error?
On 2016/11/10 00:10:12, snake wrote: > On 2016/11/10 00:02:15, commit-bot: I haz the power wrote: > > Try jobs failed on following builders: > > mac_xfa_rel on master.tryserver.client.pdfium (JOB_FAILED, > > > https://build.chromium.org/p/tryserver.client.pdfium/builders/mac_xfa_rel/bui...) > > Can i add > (void)&is_page_valid; > to fix "unused variable" error? I was hoping this would not happen, but ASSERT() is not as fancy as Chromium's DCHECK(). (void)is_page_valid; Or if you want failures to crash in release builds, use CHECK() from third_party/base/logging.h.
On 2016/11/10 00:15:52, Lei Zhang (slow) wrote: > On 2016/11/10 00:10:12, snake wrote: > > On 2016/11/10 00:02:15, commit-bot: I haz the power wrote: > > > Try jobs failed on following builders: > > > mac_xfa_rel on master.tryserver.client.pdfium (JOB_FAILED, > > > > > > https://build.chromium.org/p/tryserver.client.pdfium/builders/mac_xfa_rel/bui...) > > > > Can i add > > (void)&is_page_valid; > > to fix "unused variable" error? > > I was hoping this would not happen, but ASSERT() is not as fancy as Chromium's > DCHECK(). > > (void)is_page_valid; > > Or if you want failures to crash in release builds, use CHECK() from > third_party/base/logging.h. I do not want this in release, because it is also crash other loaded pdfs.
The CQ bit was checked by art-snake@yandex-team.ru
The patchset sent to the CQ was uploaded after l-g-t-m from thestig@chromium.org Link to the patchset: https://codereview.chromium.org/2483633002/#ps120001 (title: "fix compilation.")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
On 2016/11/10 00:20:50, snake wrote: > On 2016/11/10 00:15:52, Lei Zhang (slow) wrote: > > Or if you want failures to crash in release builds, use CHECK() from > > third_party/base/logging.h. > > I do not want this in release, because it is also crash other loaded pdfs. That sounds like a problem. Do you want to file a PDFium bug so we can keep track of the issue?
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: win on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL) win_no_v8 on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL) win_xfa on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL) win_xfa_32 on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL) win_xfa_clang on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL) win_xfa_clang_32 on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL) win_xfa_rel on master.tryserver.client.pdfium (JOB_TIMED_OUT, no build URL)
The CQ bit was checked by thestig@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
Message was sent while issue was closed.
Description was changed from ========== Do not load main cross ref table for first page in linearized pdf. For the first page in linearized PDFs, it is not necessary to load the main xref table, because it have separate special xref table in the file header. This decreasing the time to show first page in PDF plugin. ========== to ========== Do not load main cross ref table for first page in linearized pdf. For the first page in linearized PDFs, it is not necessary to load the main xref table, because it have separate special xref table in the file header. This decreasing the time to show first page in PDF plugin. Committed: https://pdfium.googlesource.com/pdfium/+/61f8e9c5aeb0d8cb5477e0248b685214746b... ==========
Message was sent while issue was closed.
Committed patchset #7 (id:120001) as https://pdfium.googlesource.com/pdfium/+/61f8e9c5aeb0d8cb5477e0248b685214746b... |