{"body":"[2020-04-15 04:49:52] Hello\n[2020-04-15 04:50:53] I'd like to contribute to the project, but would like to learn, if you are willing to accept my change, before I even start working. References:\n[2020-04-15 04:50:55] https://repo.getmonero.org/monero-project/ccs-proposals/-/merge_requests/138\n[2020-04-15 04:51:09] https://repo.getmonero.org/monero-project/ccs-proposals/-/blob/b8033720941c69a1dd45c26bf6c1bb1d03863b77/mj-compil-time-reduction.md\n[2020-04-15 04:51:42] The POC contribution can be easily done unfunded.\n[2020-04-15 05:02:28] mjxmr: i think that some parts go a bit crazy with templates (-ftemplate-depth needs to be increased). do you think it would be worth going through with clang's profiler seeing if there's anything that should be changed?\n[2020-04-15 05:04:13] → Yasmeen65Pouros joined (~Yasmeen65@ns334669.ip-5-196-64.eu)\n[2020-04-15 05:04:43] asymptotically: I think, that using profilers is always a better idea, than using a gut feeling (same applies for speed optimisations). Otherwise a programmer falls into a dogmatic way of thinking, while I prefer to work evidence based.\n[2020-04-15 05:05:09] → TheoStorm joined (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl)\n[2020-04-15 05:06:02] However in this case, boost headers, I'm absolutely sure, that this is the major slowdown. Abuse of templates create the need of having to include the boost headers in a parent header. This can be avoided with some effort.\n[2020-04-15 05:06:24] ⇐ TheoStorm quit (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl): Remote host closed the connection\n[2020-04-15 05:06:56] I am so sure, because I ate my teeth on this many times already.\n[2020-04-15 05:08:32] also what does the ccache integration with cmake do? i thought you just had to set CC and CXX\n[2020-04-15 05:08:34] ⇐ Yasmeen65Pouros quit (~Yasmeen65@ns334669.ip-5-196-64.eu): Ping timeout: 240 seconds\n[2020-04-15 05:10:04] Yes, you can do this of course, but not everybody is proficient enough to do this. My goal would be to set the CC / CXX automatically if ccache is detected in path. This is minor thing, yeah.\n[2020-04-15 05:16:46] At our job we disable ccache for special final builds, to minimize the risk of ccache affecting the binaries that go to the customer, but we have it on by default.\n[2020-04-15 05:19:09] ... during the development, I mean, so 90% of time.\n[2020-04-15 05:19:15] → TheoStorm joined (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl)\n[2020-04-15 05:28:08] → coppa joined (~Thunderbi@2a01:e0a:13c:6110:55e8:2142:245d:8808)\n[2020-04-15 05:31:34] → coppa1 joined (~Thunderbi@91-164-241-132.subs.proxad.net)\n[2020-04-15 05:31:36] ⇐ coppa quit (~Thunderbi@2a01:e0a:13c:6110:55e8:2142:245d:8808): Remote host closed the connection\n[2020-04-15 05:31:36] * coppa1 → coppa\n[2020-04-15 05:36:08] asymptotically: \"Abuse of templates create the need of having to include the boost headers in a parent header. \" A poor man's solution to this is using precompiled headers of the boost headers collection. This helps to reduce the compilation time by up to 50% per header, but a lot more this can be achieved via wrappers.\n[2020-04-15 05:39:38] ⇐ wizardsmoke quit (uid209547@gateway/web/irccloud.com/x-ixsbdjgfdwptbctr): Quit: Connection closed for inactivity\n[2020-04-15 05:40:49] Perhaps it wouldn't be a bad idea to start off with this, as the least intrusive measure?\n[2020-04-15 05:41:30] I’d wait for feedback by moneromooo / vtnerd\n[2020-04-15 05:43:36] Thanks for the tip. While I wait, I want to give mad props for cutting off the ASICs. This was a giant leap towards decentralisation.\n[2020-04-15 05:45:57] → msvb-mob joined (~msvb-lab@monero/hardware/michael)\n[2020-04-15 05:53:12] * kic0 → kico\n[2020-04-15 05:58:14] → Febo joined (59d411cd@89-212-17-205.static.t-2.net)\n[2020-04-15 06:15:51] → woodser joined (woodser@gateway/vpn/protonvpn/woodser)\n[2020-04-15 06:24:01] ⇐ favioflamingo1 quit (~Thunderbi@2405:6580:3c0:5a00:4a45:20ff:fe37:baa9): Remote host closed the connection\n[2020-04-15 06:24:05] ⇐ favioflamingo quit (~Thunderbi@2405:6580:3c0:5a00:4a45:20ff:fe37:baa9): Remote host closed the connection\n[2020-04-15 06:26:35] ⇐ Blackwolfsa4 quit (~Blackwolf@195.159.29.126): Quit: Ping timeout (120 seconds)\n[2020-04-15 06:26:38] ⇐ rdymac_ quit (~britoair@90.162.99.55): Read error: Connection reset by peer\n[2020-04-15 06:26:55] → rdymac joined (~britoair@90.162.99.55)\n[2020-04-15 06:26:59] → Blackwolfsa4 joined (~Blackwolf@195.159.29.126)\n[2020-04-15 06:27:19] * rdymac → Guest72\n[2020-04-15 06:27:37] ⇐ TheoStorm quit (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl): Quit: Leaving\n[2020-04-15 06:28:47] ⇐ redfish quit (~redfish@2a00-1dc0-2925--1e.static.as43289.net): Ping timeout: 246 seconds\n[2020-04-15 06:29:08] A speed boost would be nice. Whether it's worth churn that can introduce bugs depends on how much is gained.\n[2020-04-15 06:29:32] Last I tried, precompiled headers did not gain much, the patch end up rejected. Maybe your method might gain more though.\n[2020-04-15 06:30:02] Someone else started working on moving some serialization away from headers. Can't recall the nick now though.\n[2020-04-15 06:30:23] ccache works fine with monero, I use it all the time.\n[2020-04-15 06:30:59] I did not know CLANG had a compile time profiler. Sounds nifty.\n[2020-04-15 06:31:43] ⇐ KeiraT quit (~k4ra@gateway/tor-sasl/k4ra): Ping timeout: 240 seconds\n[2020-04-15 06:35:38] → KeiraT joined (~k4ra@gateway/tor-sasl/k4ra)\n[2020-04-15 06:41:03] ⇐ KeiraT quit (~k4ra@gateway/tor-sasl/k4ra): Ping timeout: 240 seconds\n[2020-04-15 06:41:29] ⇐ spoke0 quit (~spoke0@149.3.71.157): Remote host closed the connection\n[2020-04-15 06:41:55] → spoke0 joined (~spoke0@149.3.71.157)\n[2020-04-15 06:42:06] not sure if chromium is the only tool that can open the output :/ https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/\n[2020-04-15 06:43:17] → redfish joined (~redfish@2a00-1dc0-2925--1e.static.as43289.net)\n[2020-04-15 07:01:01] → KeiraT joined (~k4ra@gateway/tor-sasl/k4ra)\n[2020-04-15 07:12:23] moneromooo: What would you propose doing now? It might be faster to just start moving the serialization away, since I have a lot of free time until the mid-May. I don't know how I would find the contributor with over 2.200 forks currently.\n[2020-04-15 07:14:06] But I see, that there is interest. I think I'll just start then :)\n[2020-04-15 07:22:30] How about doing something else, since nobody's doing precmpiled headers AFAIK ? :)\n[2020-04-15 07:23:08] IIRC when I tried it, the reason why it did not gain much at all was we had several directories, and the precompiled thing was being done one per directory.\n[2020-04-15 07:24:05] Anyone remembers who was asking about changing serialization ? IIRC we talked about the speedup vs churn pros/cons too :)\n[2020-04-15 07:24:17] This is also cool. But when was the last time, that you heard from this person, who offered moving the serialization away from headers?\n[2020-04-15 07:24:56] Very roughly, I'd say... a couple months maybe ?\n[2020-04-15 07:25:11] because I can work full time on this now, until mid-May, meaning I could quickly bring it into life. Then just an hour or two after work.\n[2020-04-15 07:27:19] Well, your pick really.\n[2020-04-15 07:27:53] I think I'll give it a shot.\n[2020-04-15 07:28:44] Historically, I've been pushing against churn (ie, rewriting stuff for no good reason (aka some code standard says so)). I'm trying to do it less now :)\n[2020-04-15 07:29:06] So build times are a good reason, if the speedup is substantial. I'm curious to see how much you get.\n[2020-04-15 07:29:51] The point here is obviously not to create any differences in functionality.\n[2020-04-15 07:30:32] but I know from my experience, that developers usually are careless, and need a janitor :) So if you assume no churn, the mess never gets cleaned up.\n[2020-04-15 07:32:46] was it woodser ?\n[2020-04-15 07:36:19] I don't think so.\n[2020-04-15 07:36:58] ⇐ synaps3 quit (~synaps3@unaffiliated/synaps3): Ping timeout: 256 seconds\n[2020-04-15 07:53:49] ⇐ coppa quit (~Thunderbi@91-164-241-132.subs.proxad.net): Quit: coppa\n[2020-04-15 08:18:08] A small finding:\n[2020-04-15 08:18:10] enjo@tp:~/devel/github/monero-build$ make test\n[2020-04-15 08:18:11] Running tests...\n[2020-04-15 08:18:11] Test project /home/enjo/devel/github/monero-build\n[2020-04-15 08:18:12] No tests were found!!!\n[2020-04-15 08:20:23] make release-test\n[2020-04-15 08:21:40] To run one particular tests, make -C $BUILDDIR/subtree test\n[2020-04-15 08:21:54] eg: make -C build/release/foo/tests/unit_tests test\n[2020-04-15 08:23:38] make release-test\n[2020-04-15 08:23:39] make: *** No rule to make target 'release-test'. Stop.\n[2020-04-15 08:23:48] building out of source.\n[2020-04-15 08:24:48] You messed up your Makefile most likely. Check the README.md file.\n[2020-04-15 08:24:53] See git diff\n[2020-04-15 08:28:09] ⇐ smooth quit (~ubuntu@ec2-54-201-223-245.us-west-2.compute.amazonaws.com): Ping timeout: 272 seconds\n[2020-04-15 08:43:52] Ah ok. This can't be executed from an out of source directory.\n[2020-04-15 08:45:29] It can if you know what you're doing. You hinted you know cmake, so look at the contents of Makefile.\n[2020-04-15 08:46:33] it's been a couple of months since i last tried the tests but iirc it just worked when i did `cmake ..` instead of trying the makefile\n[2020-04-15 08:48:05] Let's say, that I will address this at a later time. For now I will be happy if the tests execute and keep me from screwing things up.\n[2020-04-15 08:48:11] → favioflamingo1 joined (~Thunderbi@2405:6580:3c0:5a00:4a45:20ff:fe37:baa9)\n[2020-04-15 08:48:12] → favioflamingo joined (~Thunderbi@2405:6580:3c0:5a00:4a45:20ff:fe37:baa9)\n[2020-04-15 08:49:28] ⇐ atomicman quit (~bbhgf@24-156-78-180.flagcmtk03.res.dyn.suddenlink.net): Quit: hehaw\n[2020-04-15 08:57:38] → atomicman joined (~bbhgf@24-156-78-180.flagcmtk03.res.dyn.suddenlink.net)\n[2020-04-15 08:59:42] * M5M400_ → M5M400\n[2020-04-15 09:18:25] ⇐ CommonDeer quit (~CommonDee@14-201-4-66.static.tpgi.com.au): Read error: Connection reset by peer\n[2020-04-15 09:18:51] → CommonDeer joined (~CommonDee@14-201-4-66.static.tpgi.com.au)\n[2020-04-15 09:22:42] ⇐ atomicman quit (~bbhgf@24-156-78-180.flagcmtk03.res.dyn.suddenlink.net): Remote host closed the connection\n[2020-04-15 09:41:48] → atomicman joined (~bbhgf@24-156-78-180.flagcmtk03.res.dyn.suddenlink.net)\n[2020-04-15 09:45:36] ⇐ woodser quit (woodser@gateway/vpn/protonvpn/woodser): Ping timeout: 256 seconds\n[2020-04-15 09:54:54] → woodser joined (~woodser@cam4-17.as22384.net)\n[2020-04-15 10:05:56] → nonie joined (~irc@p200300C20F131E010000000000000001.dip0.t-ipconnect.de)\n[2020-04-15 10:05:56] ⇐ nonie quit (~irc@p200300C20F131E010000000000000001.dip0.t-ipconnect.de): Client Quit\n[2020-04-15 10:15:54] ⇐ favioflamingo quit (~Thunderbi@2405:6580:3c0:5a00:4a45:20ff:fe37:baa9): Ping timeout: 256 seconds\n[2020-04-15 10:17:02] ⇐ favioflamingo1 quit (~Thunderbi@2405:6580:3c0:5a00:4a45:20ff:fe37:baa9): Ping timeout: 256 seconds\n[2020-04-15 10:39:27] moneromooo: For how long do the core_tests normally run? I feel like they hung since like an hour.\n[2020-04-15 10:42:30] Sounds plausible. If you run them often, there's a (closed) patch from me on github that speeds them up a lot.\n[2020-04-15 10:43:00] Takes ~ 10 min with the patch or so.\n[2020-04-15 10:44:40] Thank you.\n[2020-04-15 10:45:10] I will run them for the night then.\n[2020-04-15 10:46:53] Could you give me a hint, which tests cover the class tx_source_entry? I think I will start from this one as a sample small enough.\n[2020-04-15 10:47:11] Transaction tests.\n[2020-04-15 10:47:19] Which is going to be most of them.\n[2020-04-15 10:51:37] Great. Thanks!\n[2020-04-15 11:03:00] → TheoStorm joined (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl)\n[2020-04-15 11:17:35] ⇐ DeanWeen quit (~dean@gateway/tor-sasl/deanguss): Remote host closed the connection\n[2020-04-15 11:18:06] → DeanWeen joined (~dean@gateway/tor-sasl/deanguss)\n[2020-04-15 12:00:29] ⇐ TheoStorm quit (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl): Quit: Leaving\n[2020-04-15 12:01:15] ⇐ Insight quit (sid415968@gateway/web/irccloud.com/x-bbahcfjkmlxlfhtn): \n[2020-04-15 12:03:43] ⇐ maxwilliamson quit (~maxwillia@gateway/tor-sasl/maxwilliamson): Ping timeout: 240 seconds\n[2020-04-15 12:04:24] → maxwilliamson joined (~maxwillia@gateway/tor-sasl/maxwilliamson)\n[2020-04-15 12:06:23] → wizardsmoke joined (uid209547@gateway/web/irccloud.com/x-dupbbxnakvgedurx)\n[2020-04-15 12:08:20] → TheoStorm joined (~TheoStorm@host-p8vu8h.cbn1.zeelandnet.nl)\n[2020-04-15 12:14:15] mjxmr moneromooo selsta : I have a github issue related to serialization, the technique woud reduce compile times as things can be split into a cpp\n[2020-04-15 12:14:56] https://github.com/monero-project/monero/issues/6406\n[2020-04-15 12:15:00] this one?\n[2020-04-15 12:15:00] other things can be done to reduce compile times, but doing so without re-organizing large parts of the code at once coudl be tricky\n[2020-04-15 12:15:29] yes\n[2020-04-15 12:16:08] someone could reduce it further if the macro stuff was a bit much, but the general idea of splitting header/cpp is somewhat straightforward\n[2020-04-15 12:16:50] → asdc_ccc joined (~asdc_cccc@95-178-173-249.dsl.optinet.hr)\n[2020-04-15 12:17:04] the biggest compilation issues are the epee server stuff, and its going to be tricky to refactor that without it being a full rewrite (it may have to be to achieve the goa)\n[2020-04-15 12:17:39] I started some small stuff with hex encoding, trying to remove boost::lexical_cast from being included so often, its fairly heavy to pull in the inculdes\n[2020-04-15 12:18:27] gcc and clang both have a flag for reporting internal time spent compiling which is useful too\n[2020-04-15 12:18:55] the flag gives breakdowns on much time spent doing macro expansion, etc., so you can get an interna look at time spent\n[2020-04-15 12:20:28] ah found the proposal was in the other tab\n[2020-04-15 12:20:37] → sech11 joined (~sech1@31-208-56-185.cust.bredband2.com)\n[2020-04-15 12:22:05] yeah moving them into a cpp should help, but the bulk is _probably_ epee, but its hard to profile for certain due to the ways things are compiled\n[2020-04-15 12:23:47] the boost serialization stuff was separate than what I was working on, and has nearly the same problem+solution\n[2020-04-15 12:24:06] ⇐ sech1 quit (~sech1@31-208-56-185.cust.bredband2.com): Ping timeout: 265 seconds\n[2020-04-15 12:29:21] → objectorange_ joined (~objectora@172.58.19.43)\n[2020-04-15 12:32:14] ⇐ objectorange quit (~objectora@172.58.75.248): Ping timeout: 240 seconds\n[2020-04-15 12:50:53] → hv-bridge joined (~hv-bridge@web.hashvault.pro)\n[2020-04-15 12:52:02] ⇐ dethos quit (~dethos@213.190.223.7): Ping timeout: 256 seconds\n[2020-04-15 12:53:05] ⇐ derpy_bridge quit (~derpy_bri@92.223.89.201): Remote host closed the connection\n[2020-04-15 12:53:51] → derpy_bridge joined (~derpy_bri@92.223.89.201)\n[2020-04-15 13:08:30] → KeiraT- joined (~k4ra@gateway/tor-sasl/k4ra)\n[2020-04-15 13:11:23] ⇐ KeiraT quit (~k4ra@gateway/tor-sasl/k4ra): Ping timeout: 240 seconds\n[2020-04-15 13:11:26] * KeiraT- → KeiraT\n[2020-04-15 13:14:19] vtnerd: Shall we coordinate our efforts? My concrete proposal is to completely wrap the boost::archive. Putting the serialization templates can't be achieved, as it turns out, but there's no problem in wrapping the boost:archive.\n[2020-04-15 13:15:00] → dethos joined (~dethos@213.190.223.7)\n[2020-04-15 13:15:20] It will be however hard to prove improvements, since even if I wrap the b::archive in one header, it is still included by some other headers, yielding no improvement, until they are sorted out.\n[2020-04-15 13:15:47] In other words, the project needs *months* of maintenance, just be sure, that you realize it.\n[2020-04-15 13:16:33] We have to do it one by header type by one.\n[2020-04-15 13:19:21] EDIT: \"Putting the serialization templates INTO .CPP FILES can't be achieved, as it turns out\"\n[2020-04-15 13:40:33] → woodser_ joined (~woodser@cam4-17.as22384.net)\n[2020-04-15 13:42:49] ⇐ woodser quit (~woodser@cam4-17.as22384.net): Ping timeout: 264 seconds\n[2020-04-15 13:46:35] ⇐ asdc_ccc quit (~asdc_cccc@95-178-173-249.dsl.optinet.hr): Quit: Leaving\n[2020-04-15 14:13:24] ⇐ SerHack quit (~SerHack@162.210.173.127): Remote host closed the connection\n[2020-04-15 14:15:23] → SerHack joined (~SerHack@162.210.173.127)\n[2020-04-15 14:15:40] → coppa joined (~Thunderbi@2a01:e0a:13c:6110:55e8:2142:245d:8808)\n[2020-04-15 14:25:31] vtnerd: Wrapping the serialization will make it easier to integrate different serialization techniques later on.","name":"logs","extension":"txt","url":"https://www.irccloud.com/pastebin/73V4aGO4/logs","modified":1587835364,"id":"73V4aGO4","size":17601,"lines":160,"own_paste":false,"theme":"","date":1587835364}