Jekyll2022-11-29T22:09:59+00:00https://annygakh.github.io/feed.xmlAnny G’s blogSoftware development, tech industry, computer science and more.Anny GAnother blog post title 20222020-02-05T00:00:00+00:002020-02-05T00:00:00+00:00https://annygakh.github.io/dailyblog/2020/02/05/test-blog-post<h4 id="new-title">New title</h4>
<p>Title
asdasda</p>Anny GThis is really just a testMy Experience With Undergraduate Research2019-01-25T00:00:00+00:002019-01-25T00:00:00+00:00https://annygakh.github.io/longblog/2019/01/25/undergraduate-research<h2 id="introduction">Introduction</h2>
<p>In this blog post I will be talking about my experience doing undergraduate research in computer science at UBC, how I got involved with it, what I learned from it, how I approached my work, some challenges I had, and advice for students who would like to do the same. In the end I will summarize some lessons I have learned from my experiences.</p>
<ul>
<li><a href="#my-experience">My experience</a>
<ul>
<li><a href="#approaching-a-professor">Approaching a professor</a></li>
<li><a href="#choosing-a-project">Choosing a project</a></li>
<li><a href="#what-i-worked-on">What I worked on</a></li>
<li><a href="#getting-help">Getting Help</a></li>
<li><a href="#weekly-activities">Weekly activities</a>
<ul>
<li><a href="#nss-seminar">NSS Seminar</a></li>
<li><a href="#progress-reports">Progress Reports</a></li>
</ul>
</li>
<li><a href="#keeping-track-of-tasks">Keeping track of tasks</a></li>
</ul>
</li>
<li><a href="#reflection">Reflection</a>
<ul>
<li><a href="#what-i-learned">What I learned</a></li>
<li><a href="#challenges">Challenges</a></li>
</ul>
</li>
<li><a href="#advice-to-others">Advice to others</a></li>
<li><a href="#summary">Summary</a></li>
</ul>
<h2 id="my-experience">My experience</h2>
<h3 id="approaching-a-professor">Approaching a professor</h3>
<p>In the spring semester of 2018 I took <a href="https://www.cs.ubc.ca/~bestchai/teaching/cs416_2018w1/index.html">CPSC 416</a> Distributed Systems with Ivan Beschastnikh. I really enjoyed the course and wanted to learn more about distributed systems, but I wasn’t sure what my next steps could be. So I scheduled a meeting with Ivan and he gave me lots of ideas - I could take a graduate distributed systems course next year (in spring of 2019), I could experiment with building distributed applications on my own, and/or I could take a directed studies course where he would be my supervisor. I decided that I would like to do a directed studies course and so that is what I am writing about today. So what is a directed studies course? Well, it is one of the ways an undergraduate student at UBC can get a glimpse of what it’s like to do research in computer science. You can learn more <a href="https://www.cs.ubc.ca/students/undergrad/courses/specialty">here</a> about this, but the basic idea is that a student can get 3 credits, as they would for another CPSC course, and they work on either an existing research project (perhaps with other students - graduate or undergraduate) or a new research project.</p>
<h3 id="choosing-a-project">Choosing a project</h3>
<p>When it came time to choose what I would work on, I did not have a specific area I wanted to focus on, as my interests are pretty broad - instead, I just knew of a few areas I wasn’t interested in (e.g., formal verification). Ivan had sent me papers (for projects) that were submitted to either past conferences or were about to be submitted to a conference so I could get an idea of which students I should approach to find out more about projects I was interested in. I was also told that when choosing a project I should ensure that my work style was compatible with the students who are responsible for the project and that I am comfortable working with them. In addition I was advised to not hold back in asking hard questions - the kinds of questions one might ask when wanting to find out more about their team/manager before joining a new company.</p>
<p>I met with several MSc and PhD students in the lab and eventually settled on a project. Initially, I was going to work on a project related to <a href="https://www.cs.ubc.ca/~bestchai/theses/amanda-carbonari-msc-thesis-2018.pdf">“Inter-process communication in disaggregated datacenters”</a> with <a href="https://www.cs.ubc.ca/~fruffy/">Fabian</a> and it was going to be a new project picking up from Masters thesis of a former MSc student. Fabian has sent me lots of related work to read and ponder on and he also gave me access to two lab machines which I could use for my work. I also had to learn a bit about how distributed TensorFlow works and I even wrote about it <a href="https://annygakh.github.io/longblog/2018/09/13/TensorFlow.html">here</a>.
However, during one of the meetings, Ivan has brought up a <a href="https://www.usenix.org/legacy/event/usenix08/tech/full_papers/menon/menon_html/paper.html">paper</a> he has read pointing out ACK offloading into network stack as one of the contributions of the paper. Then, an idea for a new project was born. In a situation where there are two hosts and one switch, with one host sending data via switch to another host, ACK offloading could potentially increase throughput if done at the NIC (see <a href="#what-i-worked-on">“What I worked on”</a> below for more details). When I expressed interest in working on this, it was decided this work would be more useful if done as part of another project, <a href="https://ece.ubc.ca/~craigm/papers/jumpgate2018osdi.pdf">JumpGate</a>, on which another PhD student, <a href="https://ece.ubc.ca/~craigm/">Craig</a> was working on with Fabian.</p>
<h3 id="what-i-worked-on">What I worked on</h3>
<p>What does it mean for ACK offloading to take place at the NIC? The idea is that if host 1 is sending data via TCP/SCTP to host 2, the switch can send ACKs to host 1 on behalf of host 2. If ACK generation no longer depends on the CPU of the host, it leads to decreased latency and reduced variance which would be beneficial for JumpGate project, since it is about in-network data processing.</p>
<p>Luckily, I didn’t have to go through tedious network setup with multiple machines when working on ACK forwarding. To avoid having to recreate the actual network, Craig and Fabian set up a virtual network using <a href="http://mininet.org/">Mininet</a> which could be run on just one machine. The initial code at the switch in the network was very simple - if it received a packet from host 2, it would forward it to host 1 and vice versa.</p>
<p>The first prototype of ACK offloading was done in Python using <a href="https://scapy.net/">Scapy</a> in order to keep the implementation simple and understand what needs to be done for ACK offloading to work properly. To be able to construct an ACK in response to a TCP message, I, first, had to understand the TCP state machine by reading the RFC. Additionally, I inspected the contents of the TCP packets from packet captures of simple data exchange between two hosts to visualize how values of different fields in the packet change.</p>
<p>In order to gauge how slow ACK forwarding in Python + Scapy was, I timed how long it took to send a packet and get a response when the ACK forwarding was enabled vs when it was disabled, both in Python + Scapy, by parsing timestamps from packet capture files. The ACK forwarding was very slow, even when the code has undergone changes in order to make two code paths as similar as possible. This is not desirable as the entire purpose of ACK offloading is to speed up the TCP data transfer and to eliminate ACKs being the bottleneck of the TCP send rate. My next step was to implement ACK acknowledgement using <a href="https://lwn.net/Articles/740157/">eBPF</a> and <a href="https://www.iovisor.org/technology/xdp">XDP</a>.</p>
<p>eBPF and XDP allow users to insert custom code at the network stack entry point of when a packet is received or sent in order to speed up packet processing. Packets received can be sent and received on a special socket, called AF XDP, in order to bypass layers of the protocol stack.
The idea is to set up an XDP program that will send packets from the sender host to AF XDP socket #1 and to receiver host. In a C program, in response to packets from AF XDP socket #1 an ACK will be created and sent via AF XDP socket #2. The same XDP program upon receiving an ACK via AF XDP socket #2 will forward the packet to host 1. Unlike eBPF and XDP, AF XDP sockets don’t have many tutorials or examples illustrating their use, besides <a href="https://elixir.bootlin.com/linux/v4.20.3/source/samples/bpf/xdpsock_user.c">code examples</a> that live in Linux kernel repository , so Fabian has helped me go through some of the code examples and understand them. Another source of help I discovered is <a href="http://vger.kernel.org/vger-lists.html#xdp-newbies">xdp-newbies</a> - a mailing list that David Miller has created.</p>
<p>Since XDP + TCP + namespaces <a href="https://www.spinics.net/lists/xdp-newbies/msg00448.html">does not work</a> in Linux, we needed to use a different protocol, namely SCTP. I had to read the RFC about how it works and finished an implementation in Python + Scapy. The next steps are to implement ACK forwarding for SCTP using eBPF and XDP, which is currently still in progress!</p>
<p>Since Craig was going to submit a paper for JumpGate to <a href="https://www.sigops.org/2018/hotos2019/">HotOS</a> I had to pause work on ACK forwarding in December and instead work on running various benchmarks that would be included in the paper. My task was to measure the performance of various queries in Spark, as more and more preprocessing was done before the query was run. The results from running benchmarks helped motivate the need for in-network data processing. While working on my task, I have learned some specifics about Spark internals and have managed to write some Scala. Towards the submission deadline I ended up working closely with Craig when experimenting with different ways of running the benchmarks and I found it to be very rewarding and insightful. Our experiments have shown that in-network processing could improve our Spark query by up to 16x! Another exciting thing is that I am now a coauthor of a paper :) I will update this post with a link when the paper will become available.</p>
<h3 id="getting-help">Getting Help</h3>
<p>Throughout the semester I always felt that I had help available, if I ever got stuck. At the beginning of the semester, Ivan, Craig, Fabian and I had weekly meetings, where, among other things, I could discuss my progress on the project and we would talk about other ideas that can be explored. When we didn’t have weekly meetings, I would meet with Craig or Fabian and brainstorm ideas with them and ask them for help with any issues I was having.</p>
<h3 id="weekly-activities">Weekly activities</h3>
<p>After I had chosen a project to work on and submitted my application to take a directed studies course, I became a member of the <a href="https://www.cs.ubc.ca/labs/nss/html/index.html">NSS lab</a>. Being a part of the NSS lab is not just about working on a project and meeting with your supervisor and other project members. It also consists of a weekly seminar, sending out progress reports, and more. I will describe the first two in detail.</p>
<h4 id="nss-seminar">NSS Seminar</h4>
<p>Once a week a member of the NSS lab presents a paper from one of the conferences such as <a href="https://www.usenix.org/conferences/byname/179">OSDI</a>, <a href="https://www.usenix.org/conferences/byname/178">NSDI</a>, <a href="https://dl.acm.org/event.cfm?id=RE101">Eurosys</a>, <a href="http://www.sigcomm.org/">Sigcomm</a>, etc. The other members are supposed to read the paper before the seminar, and after the presentation, people evaluate the paper out loud and discuss its advantages and disadvantages. This was one of my favourite parts this semester! Even though frequently I did not understand a lot of details from the paper, I found it very valuable to hear the discussions between others because it showed me what others pay attention to when reading papers and how they evaluate papers. One time, everyone was discussing the <a href="https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-meier.pdf">NetHide</a> paper and someone asked if the same technique could be applied to code obfuscation (think JS code obfuscation) and I think this question was very fascinating because I wouldn’t have thought to compare those two techniques.</p>
<p>The way I prepared for the seminars was to read the paper the day before and try to understand the general idea behind it. During the presentation I would write down any mentioned terms/techniques I haven’t heard of before. Since that might be an overwhelming amount of things, I wouldn’t look them up right away. During the next seminar, I would do the same thing. After a few seminars I would note any terms/techniques that came up over and over again because it signified to me that it’s something I should definitely read up on and understand in greater detail.</p>
<p>Here are some example papers we looked at, one example per conference:</p>
<ul>
<li><a href="https://www.usenix.org/system/files/osdi18-lockerman.pdf">OSDI’18 - The FuzzyLog: A Partially Ordered Shared Log</a></li>
<li><a href="https://www.usenix.org/system/files/conference/nsdi18/nsdi18-tootoonchian.pdf">NSDI’18 - ResQ: Enabling SLOs in Network Function Virtualization</a></li>
<li><a href="https://www.cs.vu.nl/~herbertb/download/papers/delta-pointers_eurosys18.pdf">Eurosys’18 - Delta Pointers: Buffer Overflow Checks Without the Checks</a></li>
<li><a href="https://dl.acm.org/citation.cfm?id=3230573">Sigcomm’18 - Automated synthesis of adversarial workloads for network functions</a></li>
</ul>
<h4 id="progress-reports">Progress Reports</h4>
<p>Most of the students who have weekly meetings with Ivan are expected to send out a progress report the day before their project meeting. The point of progress reports is to communicate the work a student has done during the week, their troubles (if any) and the work they are planning to do next week. A progress report is not sent solely to the other members of the project and the supervisor, but to a dedicated mailing list. Most of the students from the NSS lab are on that mailing list. This gives everyone a chance to know about what other people are working on and sometimes even the opportunity to help someone else. Note that sending out progress reports is just one aspect of working with Ivan, and other professors will have different processes in place for students to communicate their work.</p>
<h3 id="keeping-track-of-tasks">Keeping track of tasks</h3>
<p>Keeping track of tasks when working on a research project can be a bit overwhelming, especially when there are lots of new ideas being formed during meetings. I like to keep a text document where I write down all of the tasks I need to do and what I work on each day. This helps me with writing my progress reports too, so I don’t have to struggle with remembering what I did during the week.</p>
<p>Here is an example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[ ] Run measurements
[ ] I have run into problem, ask Y about it
[X] Implement Z
[ ] Maybe we can add A, B, C to it
[ ] Since we did Z, maybe we could do E, F, G then?
</code></pre></div></div>
<h2 id="reflection">Reflection</h2>
<h3 id="what-i-learned">What I learned</h3>
<p>My project was related to networking, so I learned a great deal about TCP and SCTP protocols, network and transport layers in Linux, BPF virtual machine and XDP programs, mininet simulations, and more. But the most important thing I learned was the scientific approach to computer science. The advice I have always received from the people who were helping me with my work was - do the simplest thing to make it work in the best case scenario and you can improve it afterwards. I believe this approach is applicable not only in academic research, but also in the industry as it reinforces an idea that I think is very important - start simple! In addition to starting simple, I discovered that when trying to solve a certain problem it is important to view your work through the eyes of others and ask questions such as “why didn’t you try this simpler thing? If it didn’t solve the problem you are trying to solve or if it wasn’t efficient enough, what were its disadvantages? What’s the next simplest thing you could try to make it work?”. It’s important to consider all of these questions because it shows that you follow a scientific approach in your design.</p>
<h3 id="challenges">Challenges</h3>
<p>Sometimes I found it challenging to conceive new ideas to explore as part of my research project, and I think it’s because most of my experience consists of building things following a specification (e.g., a class assignment or a feature at work) or with a concrete end goal in mind (e.g., I would like to build an app that allows me to stream music into several bluetooth headphones or I need to build a tool to re-encrypt certain sql tables at work). I think the solution to this creative slowdown is to simply read more papers from recent conferences and to not shy away from reading papers on topics that are not of immediate interest to me, in order to build up a collection of research questions that other people ask and to understand the methodology used to answer those research questions. Despite finding it challenging, I really enjoyed having the freedom to explore various directions and experimenting with various technologies.</p>
<h2 id="advice-to-others">Advice to others</h2>
<p>In this section I will try to answer some questions that I think someone wanting to embark on the same journey as I did would find useful. These questions are a result of me trying to find out what other undergraduate students would like to know should they choose to take a directed studies course or approach a professor to do research with them. I will also try to give some advice based on my experience.</p>
<ul>
<li>Q: How do you approach a professor to ask them if you can work on something in their lab or under their supervision?
<ul>
<li>I already described how I approached my supervisor, but I must mention that I wish I had more courage to approach professors in my previous courses. I hadn’t done it earlier because I had imposter syndrome, but something that you should keep in mind is that - if you are interested in working on something and are willing to commit, you should go ahead and ask the professor about it! You might also want to approach graduate TAs, who are MSc or PhD students and would appreciate help.</li>
</ul>
</li>
<li>Q: How much flexibility does a student have in choosing a project to work on?
<ul>
<li>Sometimes students might not have the leisure of choosing which project to work on - this might be because a student hasn’t taken certain upper level classes and lacks certain experience that a project requires or it might be that a project has too many people working on it. If none of the projects in the lab are something that you want to work on, you can try to propose a new idea that can be an extension of an existing project in the lab. There are often multiple active faculty members in the same lab, whom you can approach about projects.</li>
</ul>
</li>
<li>Q: I would love to do research full time for a semester and maybe have it count as a COOP semester. Is that possible?
<ul>
<li>Yes! You might want to checkout programs such as <a href="https://www.cs.ubc.ca/students/undergrad/research-competitions/getting-involved-research">USRA</a>. You can also get awards for undergraduate research via programs such as <a href="http://science.ubc.ca/students/resources/research">SURE</a> and <a href="https://students.ubc.ca/career/campus-experiences/undergraduate-research/work-learn-international-undergraduate-research-awards">WLIUR</a>.</li>
</ul>
</li>
</ul>
<p>And here is some advice - if you have enough time and you belong to a lab where you frequently interact with other students, try to find out how you can help others with your skills. Perhaps, someone would like to experiment with an idea, tangential to their project and requires the knowledge of some niche technology that you happen to know very well and one they don’t have the time to learn and explore. This is a great opportunity to get involved and work on another project, even temporarily. However, it might not be always the case that someone will exclaim “I wish I could explore idea X using <some niche="" technology=""> but I just don’t have the time for it”, so pay attention to the obstacles others are having with their projects and try to think of ways to help out using what you know.</some></p>
<h2 id="summary">Summary</h2>
<p>Some lessons learned in no particular order:</p>
<ul>
<li>Keeping track of your work in a text document keeps you sane and on track</li>
<li>Start simple - implement the simplest thing in the best case scenario and gradually iterate on your implementation</li>
<li>Be able to explain the reasoning behind the implementation design you chose and give the cons and pros of other approaches</li>
<li>Take advantage of reading groups and try to attend as many meetings as you can</li>
<li>If you have time, figure out how to help others in the lab with their projects</li>
<li>Reading papers and learning how to evaluate them is a very important skill</li>
</ul>
<p>In conclusion, doing undergraduate research was a blast! I found NSS seminars and discussions with others valuable in helping me think critically about ideas proposed by others. I gained a lot of technical knowledge, and learned how to approach implementation of system design in an efficient and well thought-out manner. Even though I faced challenges when thinking of new directions for my research, I have outlined ways to help me become a better researcher in the future. Without making any attempt to sound less vague, to anyone who wants to solve problems in computer science, I would recommend taking a directed studies course as early as they can. And as for me, my next step is to audit an <a href="https://www.seltzer.com/margo/teaching/CS508.19/index.html">Graduate Operating Systems Course</a> to immerse myself in discussing systems research in a grad course setting.</p>Anny GIntroduction In this blog post I will be talking about my experience doing undergraduate research in computer science at UBC, how I got involved with it, what I learned from it, how I approached my work, some challenges I had, and advice for students who would like to do the same. In the end I will summarize some lessons I have learned from my experiences. My experience Approaching a professor Choosing a project What I worked on Getting Help Weekly activities NSS Seminar Progress Reports Keeping track of tasks Reflection What I learned Challenges Advice to others Summary My experience Approaching a professor In the spring semester of 2018 I took CPSC 416 Distributed Systems with Ivan Beschastnikh. I really enjoyed the course and wanted to learn more about distributed systems, but I wasn’t sure what my next steps could be. So I scheduled a meeting with Ivan and he gave me lots of ideas - I could take a graduate distributed systems course next year (in spring of 2019), I could experiment with building distributed applications on my own, and/or I could take a directed studies course where he would be my supervisor. I decided that I would like to do a directed studies course and so that is what I am writing about today. So what is a directed studies course? Well, it is one of the ways an undergraduate student at UBC can get a glimpse of what it’s like to do research in computer science. You can learn more here about this, but the basic idea is that a student can get 3 credits, as they would for another CPSC course, and they work on either an existing research project (perhaps with other students - graduate or undergraduate) or a new research project. Choosing a project When it came time to choose what I would work on, I did not have a specific area I wanted to focus on, as my interests are pretty broad - instead, I just knew of a few areas I wasn’t interested in (e.g., formal verification). Ivan had sent me papers (for projects) that were submitted to either past conferences or were about to be submitted to a conference so I could get an idea of which students I should approach to find out more about projects I was interested in. I was also told that when choosing a project I should ensure that my work style was compatible with the students who are responsible for the project and that I am comfortable working with them. In addition I was advised to not hold back in asking hard questions - the kinds of questions one might ask when wanting to find out more about their team/manager before joining a new company. I met with several MSc and PhD students in the lab and eventually settled on a project. Initially, I was going to work on a project related to “Inter-process communication in disaggregated datacenters” with Fabian and it was going to be a new project picking up from Masters thesis of a former MSc student. Fabian has sent me lots of related work to read and ponder on and he also gave me access to two lab machines which I could use for my work. I also had to learn a bit about how distributed TensorFlow works and I even wrote about it here. However, during one of the meetings, Ivan has brought up a paper he has read pointing out ACK offloading into network stack as one of the contributions of the paper. Then, an idea for a new project was born. In a situation where there are two hosts and one switch, with one host sending data via switch to another host, ACK offloading could potentially increase throughput if done at the NIC (see “What I worked on” below for more details). When I expressed interest in working on this, it was decided this work would be more useful if done as part of another project, JumpGate, on which another PhD student, Craig was working on with Fabian. What I worked on What does it mean for ACK offloading to take place at the NIC? The idea is that if host 1 is sending data via TCP/SCTP to host 2, the switch can send ACKs to host 1 on behalf of host 2. If ACK generation no longer depends on the CPU of the host, it leads to decreased latency and reduced variance which would be beneficial for JumpGate project, since it is about in-network data processing. Luckily, I didn’t have to go through tedious network setup with multiple machines when working on ACK forwarding. To avoid having to recreate the actual network, Craig and Fabian set up a virtual network using Mininet which could be run on just one machine. The initial code at the switch in the network was very simple - if it received a packet from host 2, it would forward it to host 1 and vice versa. The first prototype of ACK offloading was done in Python using Scapy in order to keep the implementation simple and understand what needs to be done for ACK offloading to work properly. To be able to construct an ACK in response to a TCP message, I, first, had to understand the TCP state machine by reading the RFC. Additionally, I inspected the contents of the TCP packets from packet captures of simple data exchange between two hosts to visualize how values of different fields in the packet change. In order to gauge how slow ACK forwarding in Python + Scapy was, I timed how long it took to send a packet and get a response when the ACK forwarding was enabled vs when it was disabled, both in Python + Scapy, by parsing timestamps from packet capture files. The ACK forwarding was very slow, even when the code has undergone changes in order to make two code paths as similar as possible. This is not desirable as the entire purpose of ACK offloading is to speed up the TCP data transfer and to eliminate ACKs being the bottleneck of the TCP send rate. My next step was to implement ACK acknowledgement using eBPF and XDP. eBPF and XDP allow users to insert custom code at the network stack entry point of when a packet is received or sent in order to speed up packet processing. Packets received can be sent and received on a special socket, called AF XDP, in order to bypass layers of the protocol stack. The idea is to set up an XDP program that will send packets from the sender host to AF XDP socket #1 and to receiver host. In a C program, in response to packets from AF XDP socket #1 an ACK will be created and sent via AF XDP socket #2. The same XDP program upon receiving an ACK via AF XDP socket #2 will forward the packet to host 1. Unlike eBPF and XDP, AF XDP sockets don’t have many tutorials or examples illustrating their use, besides code examples that live in Linux kernel repository , so Fabian has helped me go through some of the code examples and understand them. Another source of help I discovered is xdp-newbies - a mailing list that David Miller has created. Since XDP + TCP + namespaces does not work in Linux, we needed to use a different protocol, namely SCTP. I had to read the RFC about how it works and finished an implementation in Python + Scapy. The next steps are to implement ACK forwarding for SCTP using eBPF and XDP, which is currently still in progress! Since Craig was going to submit a paper for JumpGate to HotOS I had to pause work on ACK forwarding in December and instead work on running various benchmarks that would be included in the paper. My task was to measure the performance of various queries in Spark, as more and more preprocessing was done before the query was run. The results from running benchmarks helped motivate the need for in-network data processing. While working on my task, I have learned some specifics about Spark internals and have managed to write some Scala. Towards the submission deadline I ended up working closely with Craig when experimenting with different ways of running the benchmarks and I found it to be very rewarding and insightful. Our experiments have shown that in-network processing could improve our Spark query by up to 16x! Another exciting thing is that I am now a coauthor of a paper :) I will update this post with a link when the paper will become available. Getting Help Throughout the semester I always felt that I had help available, if I ever got stuck. At the beginning of the semester, Ivan, Craig, Fabian and I had weekly meetings, where, among other things, I could discuss my progress on the project and we would talk about other ideas that can be explored. When we didn’t have weekly meetings, I would meet with Craig or Fabian and brainstorm ideas with them and ask them for help with any issues I was having. Weekly activities After I had chosen a project to work on and submitted my application to take a directed studies course, I became a member of the NSS lab. Being a part of the NSS lab is not just about working on a project and meeting with your supervisor and other project members. It also consists of a weekly seminar, sending out progress reports, and more. I will describe the first two in detail. NSS Seminar Once a week a member of the NSS lab presents a paper from one of the conferences such as OSDI, NSDI, Eurosys, Sigcomm, etc. The other members are supposed to read the paper before the seminar, and after the presentation, people evaluate the paper out loud and discuss its advantages and disadvantages. This was one of my favourite parts this semester! Even though frequently I did not understand a lot of details from the paper, I found it very valuable to hear the discussions between others because it showed me what others pay attention to when reading papers and how they evaluate papers. One time, everyone was discussing the NetHide paper and someone asked if the same technique could be applied to code obfuscation (think JS code obfuscation) and I think this question was very fascinating because I wouldn’t have thought to compare those two techniques. The way I prepared for the seminars was to read the paper the day before and try to understand the general idea behind it. During the presentation I would write down any mentioned terms/techniques I haven’t heard of before. Since that might be an overwhelming amount of things, I wouldn’t look them up right away. During the next seminar, I would do the same thing. After a few seminars I would note any terms/techniques that came up over and over again because it signified to me that it’s something I should definitely read up on and understand in greater detail. Here are some example papers we looked at, one example per conference: OSDI’18 - The FuzzyLog: A Partially Ordered Shared Log NSDI’18 - ResQ: Enabling SLOs in Network Function Virtualization Eurosys’18 - Delta Pointers: Buffer Overflow Checks Without the Checks Sigcomm’18 - Automated synthesis of adversarial workloads for network functions Progress Reports Most of the students who have weekly meetings with Ivan are expected to send out a progress report the day before their project meeting. The point of progress reports is to communicate the work a student has done during the week, their troubles (if any) and the work they are planning to do next week. A progress report is not sent solely to the other members of the project and the supervisor, but to a dedicated mailing list. Most of the students from the NSS lab are on that mailing list. This gives everyone a chance to know about what other people are working on and sometimes even the opportunity to help someone else. Note that sending out progress reports is just one aspect of working with Ivan, and other professors will have different processes in place for students to communicate their work. Keeping track of tasks Keeping track of tasks when working on a research project can be a bit overwhelming, especially when there are lots of new ideas being formed during meetings. I like to keep a text document where I write down all of the tasks I need to do and what I work on each day. This helps me with writing my progress reports too, so I don’t have to struggle with remembering what I did during the week. Here is an example: [ ] Run measurements [ ] I have run into problem, ask Y about it [X] Implement Z [ ] Maybe we can add A, B, C to it [ ] Since we did Z, maybe we could do E, F, G then? Reflection What I learned My project was related to networking, so I learned a great deal about TCP and SCTP protocols, network and transport layers in Linux, BPF virtual machine and XDP programs, mininet simulations, and more. But the most important thing I learned was the scientific approach to computer science. The advice I have always received from the people who were helping me with my work was - do the simplest thing to make it work in the best case scenario and you can improve it afterwards. I believe this approach is applicable not only in academic research, but also in the industry as it reinforces an idea that I think is very important - start simple! In addition to starting simple, I discovered that when trying to solve a certain problem it is important to view your work through the eyes of others and ask questions such as “why didn’t you try this simpler thing? If it didn’t solve the problem you are trying to solve or if it wasn’t efficient enough, what were its disadvantages? What’s the next simplest thing you could try to make it work?”. It’s important to consider all of these questions because it shows that you follow a scientific approach in your design. Challenges Sometimes I found it challenging to conceive new ideas to explore as part of my research project, and I think it’s because most of my experience consists of building things following a specification (e.g., a class assignment or a feature at work) or with a concrete end goal in mind (e.g., I would like to build an app that allows me to stream music into several bluetooth headphones or I need to build a tool to re-encrypt certain sql tables at work). I think the solution to this creative slowdown is to simply read more papers from recent conferences and to not shy away from reading papers on topics that are not of immediate interest to me, in order to build up a collection of research questions that other people ask and to understand the methodology used to answer those research questions. Despite finding it challenging, I really enjoyed having the freedom to explore various directions and experimenting with various technologies. Advice to others In this section I will try to answer some questions that I think someone wanting to embark on the same journey as I did would find useful. These questions are a result of me trying to find out what other undergraduate students would like to know should they choose to take a directed studies course or approach a professor to do research with them. I will also try to give some advice based on my experience. Q: How do you approach a professor to ask them if you can work on something in their lab or under their supervision? I already described how I approached my supervisor, but I must mention that I wish I had more courage to approach professors in my previous courses. I hadn’t done it earlier because I had imposter syndrome, but something that you should keep in mind is that - if you are interested in working on something and are willing to commit, you should go ahead and ask the professor about it! You might also want to approach graduate TAs, who are MSc or PhD students and would appreciate help. Q: How much flexibility does a student have in choosing a project to work on? Sometimes students might not have the leisure of choosing which project to work on - this might be because a student hasn’t taken certain upper level classes and lacks certain experience that a project requires or it might be that a project has too many people working on it. If none of the projects in the lab are something that you want to work on, you can try to propose a new idea that can be an extension of an existing project in the lab. There are often multiple active faculty members in the same lab, whom you can approach about projects. Q: I would love to do research full time for a semester and maybe have it count as a COOP semester. Is that possible? Yes! You might want to checkout programs such as USRA. You can also get awards for undergraduate research via programs such as SURE and WLIUR. And here is some advice - if you have enough time and you belong to a lab where you frequently interact with other students, try to find out how you can help others with your skills. Perhaps, someone would like to experiment with an idea, tangential to their project and requires the knowledge of some niche technology that you happen to know very well and one they don’t have the time to learn and explore. This is a great opportunity to get involved and work on another project, even temporarily. However, it might not be always the case that someone will exclaim “I wish I could explore idea X using but I just don’t have the time for it”, so pay attention to the obstacles others are having with their projects and try to think of ways to help out using what you know. Summary Some lessons learned in no particular order: Keeping track of your work in a text document keeps you sane and on track Start simple - implement the simplest thing in the best case scenario and gradually iterate on your implementation Be able to explain the reasoning behind the implementation design you chose and give the cons and pros of other approaches Take advantage of reading groups and try to attend as many meetings as you can If you have time, figure out how to help others in the lab with their projects Reading papers and learning how to evaluate them is a very important skill In conclusion, doing undergraduate research was a blast! I found NSS seminars and discussions with others valuable in helping me think critically about ideas proposed by others. I gained a lot of technical knowledge, and learned how to approach implementation of system design in an efficient and well thought-out manner. Even though I faced challenges when thinking of new directions for my research, I have outlined ways to help me become a better researcher in the future. Without making any attempt to sound less vague, to anyone who wants to solve problems in computer science, I would recommend taking a directed studies course as early as they can. And as for me, my next step is to audit an Graduate Operating Systems Course to immerse myself in discussing systems research in a grad course setting.CppCon 2018 Trip report2018-10-14T00:00:00+00:002018-10-14T00:00:00+00:00https://annygakh.github.io/longblog/2018/10/14/cppcon2018tripReport<p>CppCon2018 ended two weeks ago and I have been unable to write about it until now. A week away from school required me to catch up on a lot of material and assignments once I got back and that’s why it took me so long to write this. While I am still not free of school stress, at least I have time to describe how I ended up at CppCon 2018, how it was and what I learned. Spoiler alert: it was life-changing! :)</p>
<h1 id="outline">Outline</h1>
<p>This post is very long, so here is an outline of Things. If you don’t have much time, I suggest you read the <a href="#summary">summary</a>. I also have handwritten (sketch?)notes <a href="/assets/pdfs/annygakhCppcon2018notes.pdf">here</a>. And here is a preview of some of my notes
<img src="/assets/images/cppcon2018/jefftrullgdbpython.jpg" alt="Image" />
<img src="/assets/images/cppcon2018/whatdowemeannothing.jpg" alt="Image" />
<img src="/assets/images/cppcon2018/simonbranddebuggers.jpg" alt="Image" />
<img src="/assets/images/cppcon2018/herbSutterMeta.jpg" alt="Image" /></p>
<ul>
<li><a href="#how-i-heard-about-cppcon">How I heard about CppCon</a></li>
<li><a href="#registering-for-cppcon-2018">Registering for CppCon 2018</a></li>
<li><a href="#about-women-in-tech-fund-and-include_cpp">About Women In Tech Fund and #include_cpp</a></li>
<li><a href="#the-weekend-before-cppcon">The weekend before CppCon</a></li>
<li><a href="#day-1---monday-sept-24-2018">Day 1 - Monday, Sept 24, 2018</a>
<ul>
<li><a href="#bjarne-stroustrup---concepts-the-future-of-generic-programming">Bjarne Stroustrup - Concepts: The Future of Generic Programming</a></li>
<li><a href="#matthew-butler---secure-coding-best-practices-your-first-line-is-the-last-line-of-defense-12">Matthew Butler - Secure Coding Best Practices: Your First Line Is The Last Line Of Defense (1/2)</a></li>
<li><a href="#lunch">Lunch</a></li>
<li><a href="#simon-brand---how-c-debuggers-work">Simon Brand - How C++ Debuggers Work</a></li>
<li><a href="#student-dinner">Student Dinner</a></li>
<li><a href="#grill-the-committee">Grill the Committee</a></li>
</ul>
</li>
<li><a href="#day-2---tuesday-sept-25-2018">Day 2 - Tuesday, Sept 25, 2018</a>
<ul>
<li><a href="#kate-gregory---what-do-we-mean-when-we-say-nothing-at-all">Kate Gregory - What Do We Mean When We Say Nothing At All?</a></li>
<li><a href="#greg-law---more-gdb-and-other-linux-debugging-wizardry">Greg Law - More gdb and other Linux debugging wizardry</a></li>
<li><a href="#richard-powell---named-arguments">Richard Powell - Named Arguments</a></li>
<li><a href="#include-cpp-dinner">#include_cpp dinner</a></li>
</ul>
</li>
<li><a href="#day-3---wednesday-sept-26-2018">Day 3 - Wednesday, Sept 26, 2018</a>
<ul>
<li><a href="#patricia-aas---software-vulnerabilities">Patricia Aas - Software Vulnerabilities</a></li>
<li><a href="#kate-gregory---simplicity">Kate Gregory - Simplicity</a></li>
<li><a href="#c-community-building-bof">C++ Community Building BoF</a></li>
<li><a href="#serge-guelton---c-in-elvenland">Serge Guelton - C++ in Elvenland</a></li>
<li><a href="#lighting-talks">Lighting Talks</a></li>
</ul>
</li>
<li><a href="#day-4---thursday-sept-27-2018">Day 4 - Thursday, Sept 27, 2018</a>
<ul>
<li><a href="#herb-sutter---thoughts-on-a-more-powerful-and-simpler-c">Herb Sutter - Thoughts on a More Powerful And Simpler C++</a></li>
<li><a href="#jeff-trull---liberating-the-debugging-experience-with-the-gdb-python-api">Jeff Trull - Liberating the Debugging Experience with the GDB Python API</a></li>
<li><a href="#my-lighting-talk">My Lighting Talk</a></li>
</ul>
</li>
<li><a href="#day-5---friday-sept-28-2018">Day 5 - Friday, Sept 28, 2018</a></li>
<li><a href="#summary">Summary</a>
<ul>
<li><a href="#reflection-1">Reflection</a></li>
<li><a href="#other-random-good-things-that-happened-to-me-in-no-particular-order">Other Random Good Things That Happened To Me In No Particular Order</a></li>
<li><a href="#thank-you">Thank You</a></li>
</ul>
</li>
</ul>
<h1 id="how-i-heard-about-cppcon">How I heard about CppCon</h1>
<p>C++ is one of my favourite programming languages and I like anything to do with C++. Around two years ago I discovered CppCon conference videos online, and have watched quite a few talks since then. Then I thought it must be so cool to attend this conference, watch speakers present different things one can do in C++ and meet people who are interested in the same things as you.</p>
<h1 id="registering-for-cppcon-2018">Registering for CppCon 2018</h1>
<p>This August, I was looking to see which conferences were happening nearby me and to my surprise I have realized that CppCon has always been in Bellevue (I am in Vancouver, Canada) and it was going to be held quite soon! Then I discovered that <a href="https://womenintechfund.org">Women In Tech Fund</a> was accepting applications from women who are in need of financial assistance to be able to attend the conference. I wrote about how I would benefit from attending such conference and what I hope to get out of it and submitted my application.</p>
<p>The next day I have received a reply saying that my application has been accepted. Wow! It was a really happy day for me. After some time I was contacted by the organizers of the CppCon and they helped me register for CppCon and ensured that I had a place to stay for the duration of the conference.</p>
<h1 id="about-women-in-tech-fund-and-include_cpp">About Women In Tech Fund and #include_cpp</h1>
<p>I would like to share some information about <a href="https://womenintechfund.org">@WomenInTechFund</a> and <a href="https://www.includecpp.org/">@include_cpp</a> because I think those are awesome organizations/communities that everyone should know about!</p>
<p>First, a little bit about Independent Fund for Women in Tech (@WomenInTechFund)</p>
<blockquote>
<p>The Independent Women in Tech Fund aims to help women attend security conferences by providing assistance with entry ticket and possible travel support. […]
For most women, the opportunity to attend conferences, talk to peers, see other women in the field, ask for advices and feel that they are supported makes a difference in their careers. Such support may help decide their future. We want to help with that.</p>
</blockquote>
<p>And a little bit about #include_cpp</p>
<blockquote>
<p>#include<C++> is a global, inclusive, and diverse community for developers interested in C++. Here, you can find a welcoming space to learn and discuss C++. We also provide resources to create safer, more inclusive, community gatherings.</p>
<p>We believe that a community is only as good as how it treats its most vulnerable members. Therefore, we strive to create a welcoming, safe, and accessible environment. You can find our code of conduct <a href="https://www.includecpp.org/code-of-conduct">here</a>.</p>
</blockquote>
<p>So what’s going on? :) What happened and how did the two organizations work together to help women (cis and trans) to attend CppCon? To summarize the story from the <a href="https://www.gofundme.com/sponsoring-women-CppCon-a">gofundme campaign</a> - #include_cpp has been in contact with the Women in Tech Fund and CppCon organizers, and were raising money for women to attend this conference. And Women In Tech Fund was administering the tickets.</p>
<p>43 people have donated to this cause! That is super awesome!</p>
<h1 id="the-weekend-before-cppcon">The weekend before CppCon</h1>
<h2 id="reaching-out-to-people-on-twitter">Reaching out to people on Twitter</h2>
<p>I wanted to do a shout out to Women In Tech Fund, #include_cpp and the 43 donors, so I thanked them on Twitter (since all the cool people are hanging out on Twitter) and then my tweet got a lot of retweets! People were very nice and told me</p>
<ul>
<li>they were glad I get a chance to attend CppCon</li>
<li>they wanted to meet me and I should find them when I have a chance</li>
<li>they would gladly introduce me to other people</li>
</ul>
<p>I also reached out myself to a few other people who were tweeting under #IGotYourBack, and I tweeted to them thanking them and saying that I hope I get to meet them at the conference.</p>
<h2 id="deciding-which-talks-to-attend">Deciding which talks to attend</h2>
<p>Now that I was going to CppCon for sure, I needed to decide which talks I should attend. It was. Really. Overwhelming. I wrote about it <a href="http://annygakh.github.io/longblog/2018/09/22/prereadingCppCon2018talks.html">here</a>. To summarize - I went through the description of each talk carefully, and wrote down what I know already in relation to the topic and what I hope to learn about it. I also tweeted about it and people liked it as well!</p>
<h1 id="day-0---sunday-sept-23-2018">Day 0 - Sunday, Sept 23, 2018</h1>
<h2 id="cpp-tee-shirt-dinner">Cpp Tee Shirt Dinner</h2>
<p>I call it day 0 because the official talks did not begin until the next day. My plan for the day was to bus to Bellevue from Vancouver, to attend a Cpp Tee Shirt Dinner and to attend a reception. From what I understand, this dinner is sort of like a tradition, and you are supposed to wear a Cpp tshirt and go to one of the pre-selected restaraunts, alone or with a group. This gives people a chance to mingle and meet new people. Since I didn’t know anybody, I was really excited to attend the dinner. Also, on twitter, CppCon folks were sharing the names of the restaunts they were planning to head to, which was helpful. In the morning @pati_gallardo reached out to me, which was very very nice of her, and I decided I would go to the same restaraunt as her so I could meet her in person! I went there and met her and quite a few other people (spoiler alert # 1: they were super awesome people) and I had such a great time. I got to know some folks, where they work, what they work on and etc. We sat there for two hours or so and more and more people were joining, which was awesome!</p>
<h2 id="reception">Reception</h2>
<p>After the dinner there was a reception back in the convention center and we went there. It was Patricia’s first time at the CppCon, but she already knew quite a few people and she made sure to introduce me to everyone she knew and anyone else that I wanted to meet. That. Was. Super. Fun! Everyone was so <strong>nice</strong> and interesting to talk to! Quite a few of them were folks I briefly interacted on Twitter. I had good conversations with people, learned new things (e.g. monads in C++) and met some people whose work I have seen before - e.g. Rob from @cppcast!!! what!! I used to listen to that podcast all the time while commuting to university. I also chatted with @hankadusikova and @michaelcaisse and they convinced me to do a lighting talk (spoiler alert #2: I did it)!</p>
<h2 id="reflection">Reflection</h2>
<p>I came back to my hotel and couldn’t stop smiling because I was really happy. I couldn’t believe how welcoming and nice everyone was. I got to participate in so many interesting conversations and no one acted condescending if I asked something that everyone else might have already known - quite the opposite - they were very eager to explain.</p>
<h1 id="day-1---monday-sept-24-2018">Day 1 - Monday, Sept 24, 2018</h1>
<p>Finally, I will talk about the talks I have seen. But first, a little disclaimer - I’m going to talk about what I took away from the talks. Sometimes the talks went over my head and I was only able to write down keywords and ideas I need to read up on. I think that’s okay! I also took lots of notes (with doodles of course) and I posted them on twitter and will be attaching them here as well. A lot of info that is contained in my notes will be repeated here because I’m looking over my notes and typing them up here into cohesive paragraphs.</p>
<h2 id="bjarne-stroustrup---concepts-the-future-of-generic-programming">Bjarne Stroustrup - Concepts: The Future of Generic Programming</h2>
<p>This talk was not really oriented at technical language details. Bjarne said that individual features in isolation from the language are not the intent here. The goal of the talk was to improve generic programming so it becomes as simple as non-generic code, but to achieve that, we need to improve other features.</p>
<p>One might try to find similarities between Types and Concepts. Roughly put, Types specify the set of operations for an object. With Concepts, however, you specify how one can use an object, and it is not just for functions or classes.</p>
<p>Some advantages of concepts include:</p>
<ul>
<li>readability</li>
<li>maintainability</li>
</ul>
<p>Great! Let’s look at some properties of Concepts:</p>
<h4 id="some-properties-of-concepts">(Some) Properties of Concepts</h4>
<p><strong>Conditional properties</strong></p>
<p>This allows you to express the following idea in the code “make it so -> operator can only be used if other is a class”. Hold on, can’t we use <code class="language-plaintext highlighter-rouge">std::enable_if</code> for this? Well no, they don’t really scale and lots of workarounds are needed.</p>
<p><strong>Reliability</strong></p>
<p>Sometimes <code class="language-plaintext highlighter-rouge">auto</code> is misused, like in the following example</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">auto</span> <span class="n">x</span> <span class="o">=</span> <span class="n">foo</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="c1">// x is InputChannel</span>
</code></pre></div></div>
<p>The above code is not really maintainable and the comment contains a requirement that can’t be enforced by the compiler.</p>
<p>Fear not, because with Concepts you can do this:</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">InputChannel</span> <span class="n">x</span> <span class="o">=</span> <span class="n">foo</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</code></pre></div></div>
<p><strong>+</strong> This is readable and can be enforced by the compiler.</p>
<h4 id="concept-of-concepts">Concept of Concepts</h4>
<p>Concepts aren’t really a new thing! Think of <strong>compile time predicates</strong> - they are quite fundamental and we have had them for a while. Examples include</p>
<ul>
<li>developer’s head</li>
<li>documentation</li>
<li>code comments</li>
</ul>
<p>But now, we get to represent those ideas in code! To summarize, I will list (some) of the pros again:</p>
<p><strong>+</strong> Concepts are flexible and allow you to keep extending and working on them (because you probably won’t get it right the first time)</p>
<p><strong>+</strong> Once you use concepts you won’t go back because you begin thinking in terms of them</p>
<p><strong>+</strong> You express ideas more clearly so <strong>even a compiler can understand</strong></p>
<p>This change is not a detail, but a change in how we think, so it is a major change, and that’s is why it might take a while to get adopted.</p>
<h4 id="thoughts">Thoughts</h4>
<p>I thought this talk was informative and now I understand the appeal and importance of Concepts. I also found this talk interesting because Bjarne stressed how this change is quite fundamental and it will change how we will think about it.</p>
<h2 id="lunch">Lunch</h2>
<p>I thought it was important to talk about how my lunch was, given that I was very worried about being alone at this conference. I joined the #include_cpp discord on Sunday and I thought I would ask on the #CppCon channel if anyone wanted to get lunch together. Folks from #include_cpp had a booth for first two days of the conference, and so it happened that we would gather there at lunch time and head to some food place. It was very nice and I enjoyed speaking to people in informal settings about what they work on and what they do, and to get to know more people from the community.</p>
<p>I also thought it was very cool that at first it would be a small group of us waiting to go out for lunch and then 10 minutes later when we are actually heading out of the convention center, 5 more people have joined.</p>
<h2 id="matthew-butler---secure-coding-best-practices-your-first-line-is-the-last-line-of-defense-12">Matthew Butler - Secure Coding Best Practices: Your First Line Is The Last Line Of Defense (1/2)</h2>
<p>So, unfortunately, I was only able to go to the first part of the talk, because during the second part of the talk I really wanted to attend Simon Brand’s talk on ‘How C++ Debuggers work’, but, I think I learned quite a few new things even from only one part of the talk. So here are my notes!</p>
<h4 id="intro">Intro</h4>
<p>So some companies might think that they don’t really need to care much about security. Here are 3 Reasons Why A Company Thinks They Are Safe From Attackers</p>
<ul>
<li><strong>presence of firewalls</strong> - unfortunately, it alone did not save other companies.</li>
<li><strong>code review and tests</strong> - code reviews are not focused on finding exploits</li>
<li><strong>we are too {small,big,etc} to be a target</strong> - if you are a big company, you have lots to lose, and if you are a small company, your security is (most likely) too weak. As a result, you are always a target.</li>
</ul>
<h4 id="terms">Terms</h4>
<p>It might be helpful to define what a <strong>critical system</strong> is. A critical system is any system, regardless of its priority, that any other systems can interact with. An example is a wireless printer. Matthew told a story from his life of a time when a printer in the lab(?) of someone he knew was used to infiltrate the system.</p>
<p>Another term that needs to be defined is <strong>attack vector</strong>. Attack vectors are anything that one from outside can get into, e.g. command line interfaces.</p>
<h4 id="techniques">Techniques</h4>
<p>So, how do you make your programs safer? You need to remember to</p>
<ol>
<li>Validate incoming arguments</li>
<li>Study the standard</li>
<li>And remember <strong>warnings are errors</strong>. Listen to them!</li>
</ol>
<p>While these are not comprehensive, it might also help to know some of the things that Matthew Butler will look for when penetrating a system:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">copy</code>/<code class="language-plaintext highlighter-rouge">moves</code> of memory</li>
<li>not validating input data or verifying the sender</li>
<li>use of open source libraries
<ul>
<li>their weaknesses are your weaknesses</li>
</ul>
</li>
<li>unguarded internal interfaces
<ul>
<li>don’t assume inside systems to be safe!</li>
</ul>
</li>
<li>complexity in code design</li>
</ul>
<h4 id="thoughts-1">Thoughts</h4>
<p>It was very interesting! There was also an explanation (or a live demo) of buffer overflows, and I always find those to be fun. I wrote down some things I should look up again, because I forgot the details of how they work</p>
<ul>
<li>ASLR - randomizing address space</li>
<li>Stack canaries - putting known bit patterns that needs to be checked before returning from a function. The code can abort if the canary is incorrect. And all of this is enforced by the compiler!</li>
<li>Return oriented programming</li>
</ul>
<h2 id="simon-brand---how-c-debuggers-work">Simon Brand - How C++ Debuggers Work</h2>
<h4 id="elf--and-dwarf">ELF and DWARF</h4>
<p><strong>ELF</strong> is a binary format for executable and it has headers and other useful things. There is also another format <strong>DWARF</strong> and it is a <em>debug info format</em>. It describes stuff about your program, e.g. what functions you have and what types they return, what types you have, etc. Debuggers usually consume DWARF, but sometimes ELF is enough for your debugger but usually for complicated things your debugger needs more info.</p>
<h4 id="breakpoints">Breakpoints</h4>
<p>So what are some things that come to your mind when you think of debuggers? I think of breakpoints. Turns out there are hardware breakpoints and software breakpoints.</p>
<p><strong>Hardware breakpoints</strong> have special registers, but the number of them is limited. They can break on <code class="language-plaintext highlighter-rouge">read</code>/<code class="language-plaintext highlighter-rouge">write</code>/<code class="language-plaintext highlighter-rouge">execute</code> of a certain address. In <code class="language-plaintext highlighter-rouge">x86</code> there are only 4 registers and I think you have to write the breakpoint address into those registers.</p>
<p><strong>Software breakpoints</strong> are implemented in software. The running code is modified so that a breakpoint occurs. You can have an unlimited number of breakpoints. A limitation is that they can only break on <code class="language-plaintext highlighter-rouge">execute</code>. In <code class="language-plaintext highlighter-rouge">x86</code> you have to replace the current instruction with <code class="language-plaintext highlighter-rouge">int3</code>.</p>
<h4 id="other-random-bits-and-bobs">Other random bits and bobs</h4>
<p>These are some definitions and notes I took down regarding some concepts. They are a bit messy and have no story connecting them because some of this stuff went over my head :)</p>
<ul>
<li><strong>mangling</strong> is cool! each function gets mangled differently and the process differs from compiler to compiler.</li>
<li><code class="language-plaintext highlighter-rouge">process_vm_ready</code> reads more than one page at a time??</li>
<li><strong>DW-tag subprogram</strong> - each function in DWARF has entries for its beginning and end. This allows you to compare current PC to ranges of each function and discover the exact name of the current function that is being processed.</li>
<li>DWARF gives you info about inlined functions.</li>
<li>things related to **shared libraries **
<ul>
<li><strong>r_debug</strong></li>
<li><strong>link_map</strong> - tells you of all shared libraries</li>
<li><strong>r_brk__</strong> - gets called when a library is being called</li>
</ul>
</li>
</ul>
<h4 id="thoughts-2">Thoughts</h4>
<p>This was a super interesting talk( and topic) and my plan for when I have some more time is to read about all of this in greater detail. During the talk I had some questions that I’m sure I can answer once I start researching some more (or rewatch the talk)</p>
<ol>
<li>How is DWARF generated?</li>
<li>What exactly happens when an instruction is replaced with <code class="language-plaintext highlighter-rouge">int3</code>?</li>
<li>How do line breakpoints work? What’s the process of generating those?</li>
<li>How does <code class="language-plaintext highlighter-rouge">step out</code> and <code class="language-plaintext highlighter-rouge">step in</code> work?</li>
</ol>
<h2 id="student-dinner">Student Dinner</h2>
<p>For dinner, all of the full-time current students who were attending CppCon were invited to a dinner with other students and some famous people from C++ community (like Bjarne S.). I had a lot of fun! I met some other students and learned about the kinds of things they do with C++ and it was very interesting.</p>
<h2 id="grill-the-committee">Grill the Committee</h2>
<p>After the dinner there was ‘Grill the Committee’ session. Even though up until this point I haven’t really kept up with what is going on in the C++ community or different proposals, I learned about some ongoing concerns that people have or some areas of C++ that are of great interest/concern to many.</p>
<h1 id="day-2---tuesday-sept-25-2018">Day 2 - Tuesday, Sept 25, 2018</h1>
<h2 id="kate-gregory---what-do-we-mean-when-we-say-nothing-at-all">Kate Gregory - What Do We Mean When We Say Nothing At All?</h2>
<p>This talk poached the reader to think about what the presence of certain keywords in the code communicates to the readers and how to ensure that lack thereof is also meaningful.</p>
<p>This is a quote that I really liked from Kate -</p>
<blockquote>
<p>You should be writing your code as if your children will be maintaining it.</p>
</blockquote>
<p>In other words - ensure you are communicating everything you want with the code you are writing. Kate gave lots of advice on how to communicate your intentions via different constructs.</p>
<h4 id="fallthrough">[[fallthrough]]</h4>
<p>Using <code class="language-plaintext highlighter-rouge">fallthrough</code> does not make a difference at runtime, but the compiler will complain if you misuse it.</p>
<h4 id="on-defaults">on defaults</h4>
<ul>
<li>Avoid defaults because not everyone knows it. Not using defaults allows you to communicate your intent better.</li>
<li>Saves others time guessing if you have considered it</li>
<li><strong>For nothing to mean something, in other places you have to make your intent <em>explicit</em></strong></li>
</ul>
<h4 id="on-parameter-passing">on parameter passing</h4>
<p><code class="language-plaintext highlighter-rouge">const</code>, <code class="language-plaintext highlighter-rouge">&</code> convey the meaning about your API design (examples in the talk)</p>
<h4 id="on-traditional-loops">on traditional loops</h4>
<p>When you have a <em>traditional loop</em>, is it because it’s not touching some objects? If yes, well there are algorithms for that… There are lambdas too.</p>
<h4 id="importance-of-communicating">Importance of communicating</h4>
<ul>
<li>Clear code communicates with future readers</li>
<li>Shows them why you did something (your intent)</li>
<li>“What does it mean when you make such choices”</li>
</ul>
<p><strong>Ensure your nothing speaks volumes</strong>.</p>
<h4 id="thoughts-3">Thoughts</h4>
<p>I really like any talk that Kate gives. Communicating with future readers of your code is really important, and Kate gives lots of techniques for ensuring effective communication.</p>
<h2 id="greg-law---more-gdb-and-other-linux-debugging-wizardry">Greg Law - More gdb and other Linux debugging wizardry</h2>
<p>Soo I got a bit confused during this talk so my notes are just bullet points of some bits of information I understood and things I will look up in future</p>
<h4 id="intro-1">Intro</h4>
<p>There are two main classes of debugging tools:</p>
<ol>
<li>Checkers
<ul>
<li>Does my code do a bad thing, X, Y, Z? Examples include valgrind, address sanitizers, etc.</li>
</ul>
</li>
<li>Debuggers</li>
</ol>
<h4 id="signals">Signals</h4>
<ul>
<li>When the program receives a signal, it’s suspended and tracer(??) gets notified</li>
<li><code class="language-plaintext highlighter-rouge">gdb</code> might decide not to handle the signal</li>
<li><code class="language-plaintext highlighter-rouge">handle SIGSYS stop print pass</code> is how you can tell <code class="language-plaintext highlighter-rouge">gdb</code> to not handle certain signals</li>
</ul>
<h4 id="misc">Misc</h4>
<ul>
<li>Valgrind is a platform on top of which other checkers, e.g. memcheck, are built. It also has a remote debug server.</li>
<li>There are different kind of sanitizers - address, memory, leak, etc…</li>
</ul>
<h4 id="things-to-look-up-in-detail">Things to look up in detail</h4>
<ul>
<li>ftrace - function tracer</li>
<li>strace - traces all function calls of a process</li>
<li>ltrace - dynamic calls of a process</li>
<li>reverse-step in gdb??</li>
<li>live recorder in gdb??</li>
</ul>
<p>I know that *trace are very useful tools to know about, but I just never happened to use them before. I think I used strace for something once, but that’s it. I definitely need to look up all of those tools to see what kinds of things are possible to use them for.</p>
<h4 id="thoughts-4">Thoughts</h4>
<p>I am interested in the things presented in this talk, so when I have more time I will definitely rewatch this.</p>
<h2 id="richard-powell---named-arguments">Richard Powell - Named Arguments</h2>
<p>In this talk, Richard showed the process of turning an idea about C++ language extension into reality. This was one of my favorite live demos! Richard incrementally prototyped named arguments in C++ from scratch.
At first, he started with posing the question - what if we wanted to go from <code class="language-plaintext highlighter-rouge">foo(6)</code> to <code class="language-plaintext highlighter-rouge">foo(a=6)</code> in C++? How would we start? Well, what if we had a map with arguments and their names? Then we could extract arguments from the map and feed them to the function.</p>
<p>The high level steps are:</p>
<ul>
<li><strong>construct</strong> the arguments</li>
<li><strong>extract</strong> the arguments</li>
<li><strong>unpack</strong> the arguments</li>
</ul>
<p>I really liked the way he structured the presentation and live demo. It was not confusing at all, even though I didn’t know hana boost library or some of the exact techniques that Richard used.</p>
<p>At the end of the talk, Richard emphasized that it’s okay that this example is not production ready. The important thing that he wanted to demonstrate is how turn an idea into a product. Not everything can be production ready, but the important thing is <strong>journey</strong> and <strong>learning things</strong>. One of the things he left us with was “Find a question. Then ask yourself, how can you do this in/with C++? “</p>
<p>I found this talk to be very insightful on how to approach problems in C++. I learned a lot and I really liked that Richard emphasized the journey and learning things.</p>
<h4 id="things-to-look-up">Things to look up</h4>
<ul>
<li>hana boost library</li>
<li>hana literals</li>
<li>What does header only library mean? Is the code only located in the header? Is that it?</li>
</ul>
<h2 id="include_cpp-dinner">#include_cpp dinner</h2>
<p>Another cool thing happened - I got to go to a #include_cpp dinner. There were a lot of people from #include_cpp community. There was dinner and a panel discussion featuring prominent women from the C++ community. They discussed how they got started with C++, what challenges they faced and gave advice. The dinner was great - we ate good food and chatted with people at our table. I got to know some other women in C++ and what they work on. Then, the panel discussion began after dinner. I’m not sure how to describe it, but I teared up a lot and wanted to give everyone a hug. I’m so glad to have heard from all of those women. Hearing about their experiences made me aware that my experiences are not unique and it was helpful to hear their advice on dealing with various challenges they faced. I’m so grateful that they shared all of those things with us. I’m also very grateful that this dinner was organized, because it was definitely one of my favorite events during this week.</p>
<h1 id="day-3---wednesday-sept-26-2018">Day 3 - Wednesday, Sept 26, 2018</h1>
<h2 id="patricia-aas---software-vulnerabilities">Patricia Aas - Software Vulnerabilities</h2>
<p>Patricia’s talk focused on software vulnerabilities. Some of the lessons I learned</p>
<ul>
<li>Don’t depend on undefined behavior</li>
<li>Sometimes compilers optimize away the <code class="language-plaintext highlighter-rouge">memset</code>
<ul>
<li>Can be an issue if you are depending on it to erase your passwords!!!</li>
</ul>
</li>
<li>Shellcode - piece of code. Today it is meant to represent the fact that you are able to execute something natively</li>
<li>Don’t take external strings as format strings. They might not supply enough args and printf will read whatever the next thing on stack is</li>
</ul>
<p>Patricia also showed how to perform a buffer overflow attack. I couldn’t help but notice how well organized the code examples were. I think everyone who wants to show code on their slides needs to follow Patricia’s examples. She highlighted different parts of the assembly or C code that we needed to look at, and it made it easier to follow along and not get distracted by other lines of code!</p>
<p>Patricia also talked about X Things She Would Rather You Not Do to prevent certain vulnerabilities. I did not write them down because I was afraid I would not be able to explain them properly. So you will have to watch the talk :)</p>
<h2 id="kate-gregory---simplicity">Kate Gregory - Simplicity</h2>
<p>I got to introduce Kate for her keynote! Whaat? How did that happen? On Tuesday morning, Jon Kalb has emailed me and said that he really liked my <a href="TODO">blog post</a> that Bryce forwarded to him. He asked if I would like to introduce Kate for her keynote. I couldn’t believe it! As I have previously mentioned, I am a huge fan of Kate, and I have watched a lot of her videos. So of course I said yes! Before I went on stage, Jon gave me some advice “Breathe. Smile. Speak slowly”, which I definitely tried to follow. It was such an honor to introduce Kate! :’)</p>
<p>Here is a list of lessons learned (but of course, please exercise your judgement):</p>
<ul>
<li>When you teach, start simple. Omit things like error checking in order to lesson the cognitive burden.</li>
<li>Remember, encapsulation != obscuration.</li>
<li>Simpler code is harder to write! It is not often faster, because to make it faster you have to know somethign about the language. E.g. <code class="language-plaintext highlighter-rouge">auto p</code> vs <code class="language-plaintext highlighter-rouge">auto p&</code>.</li>
<li>Short function names! If you can’t name it, it is probably several functions.</li>
<li>No nesting, return from your function early. (Also, there was a lighting talk on Wednesday or Thursday called ‘Shape of a program’ by @JamesMcNellis dedicated to this topic alone. Highly entertaining and also useful.)</li>
<li>Clean up in destructors. No one will forget to clean up, especially if they add a new <code class="language-plaintext highlighter-rouge">throw</code> statement somewhere in one of the functions.</li>
<li><strong>Think about the future</strong>. It might be easy now, but later it will confuse people.</li>
</ul>
<h2 id="c-community-building-bof">C++ Community Building BoF</h2>
<p>This was a Birds of a Feather session. The goal was to discuss how to build a successfull C++ community. I learned a lot of things and tried to write all of them down.</p>
<h4 id="running-meetups">Running meetups</h4>
<p>If you would like to organize a meetup, here are some ideas for how to do so:</p>
<ul>
<li>Reach out to speakers in other cities, in case they are travelling. If they are visiting your city, politely ask if they would like to speak at your event.</li>
<li>It’s important not only to have talks but also discussions and have the meetup be a group and not just a service with talks.</li>
<li>Have lighting talks</li>
<li>What about organizing a book club for various C++ books/articles?</li>
<li>Get people to showcase their projects.</li>
<li>What about reviewing resumes and giving people some feedback?</li>
<li>Sometimes have activities that allow everyone to break up into smaller groups</li>
<li>Ensure diversity of topics. E.g. if you find yourself mostly presenting talks about debuggers, maybe think about presenting something about graphics in C++. Or add some non-technical talks.</li>
</ul>
<h4 id="sponsorship">Sponsorship</h4>
<p>How should you talk to a company when you would like their sponsorship?</p>
<ul>
<li>Tell them what they will get in return
<ul>
<li>e.g. you can provide them with 2 mins at the beginning of each talk, or distribute their swag, etc</li>
</ul>
</li>
<li>Tell them concrete things that you will need
<ul>
<li>e.g. pizza and printed flyers</li>
</ul>
</li>
<li>Maybe look for sponsorships from companies who hire C++ developers, because they will have more incentive to help you</li>
</ul>
<h4 id="voluntering">Voluntering</h4>
<p>How do you get volunteers to help you with your meetup?</p>
<ul>
<li>Split up and rotate roles between people.</li>
<li>Can assign different people to find speakers.</li>
<li>Ask for specific things to get done
<ul>
<li>e.g. we need someone to order pizza, receive the delivery and setup the pizzas</li>
</ul>
</li>
<li>You have to let go and not micro-manage people!</li>
<li>Very important - appreciate volunteers - people are helping you in their free time, for free.</li>
</ul>
<h2 id="serge-guelton---c-in-elvenland">Serge Guelton - C++ in Elvenland</h2>
<p>Another really cool talk I wish I could understand but it went over my head :( That’s okay though! I wrote down things I was confused about and I can read up on them later, and rewatch the talk.</p>
<ul>
<li>Symbol tables in ELF</li>
<li>Symbol demangling</li>
<li>Shrinking existing libraries</li>
<li>UND in symbol table</li>
<li><code class="language-plaintext highlighter-rouge">nm</code></li>
<li><code class="language-plaintext highlighter-rouge">readelf</code></li>
<li><code class="language-plaintext highlighter-rouge">objdump</code></li>
<li><code class="language-plaintext highlighter-rouge">objcopy</code></li>
</ul>
<h2 id="lighting-talks">Lighting Talks</h2>
<p>Lighting talks are such a great idea. @michaelcaisse is a very talented and super hilarious host.</p>
<p>One thing I have written down from the talks that I would like to look up is <code class="language-plaintext highlighter-rouge">std::basic_string</code>. Someone presented a lighting talk on using it instead of a vector. I’m intrigued.</p>
<h1 id="day-4---thursday-sept-27-2018">Day 4 - Thursday, Sept 27, 2018</h1>
<h2 id="herb-sutter---thoughts-on-a-more-powerful-and-simpler-c">Herb Sutter - Thoughts on a More Powerful And Simpler C++</h2>
<p>This was a long talk, but my notes are succint because in the middle of it there was a demo. I really like the way the demo was done. It got the point across as one wants a demo to do, but without the panics that everything might crash and not work. I thought this talk was well presented, and even though I have not heard of meta classes before, Herb explained them well and I was able to understand their purpose and why they are a Good Thing. Below are some notes I took.</p>
<p>When we add new things to the language, it is to</p>
<ol>
<li>Make user code simpler</li>
<li>Remove edge cases, e.g. lifetimes</li>
<li>Discourage certain things</li>
</ol>
<p>How do we know which features represent C++. Or to be more clear, what is C++? **</p>
<ul>
<li>Don’t pay for what you don’t use (0 overhead). Of course, you still have to pay for it when you use it.</li>
<li>Leave no room for lower language (except for assembly).</li>
<li>Backwards compatibility</li>
</ul>
<p>What are properties of Good C++ Features?</p>
<ul>
<li>Express intent better</li>
<li>Simplify C++ code (read, write, debug)</li>
<li>Remove an exception to **</li>
<li>Remove gotchas</li>
<li>Help us deprecate something (e.g. <code class="language-plaintext highlighter-rouge">typedef</code> -> <code class="language-plaintext highlighter-rouge">using</code>)</li>
</ul>
<h2 id="jeff-trull---liberating-the-debugging-experience-with-the-gdb-python-api">Jeff Trull - Liberating the Debugging Experience with the GDB Python API</h2>
<p>This was one of my favorite talks!I was so blown away by this talk. I have previously heard of GDB’s python APIs but I never got a chance to play around with them. The documentation was kind of scary to me and I wasn’t really sure what is possible.</p>
<p>Jeff’s presentation was great! His live demos were well prepared and they were truly necessary in his talk. This talk not only taught us about Python APIs, but also taught us how to approach any obstacles in gdb and what is possible. I learned a lot of things and as soon as I have some time, I will be playing around with those APIs. I can’t wait to build python hooks to make my debugging easier.</p>
<p>Here are 3 Very Cool Things You Can Do With GDB Python APIs.</p>
<h4 id="1-improve-stack-traces">1. Improve Stack Traces</h4>
<p>Frame decorators are filters and you can override them to add custom behavior. Jeff showed us how we could replace verbose <code class="language-plaintext highlighter-rouge">std</code> names with well known values.</p>
<h4 id="2-easily-step-to-user-code">2. Easily Step To User Code</h4>
<p>I don’t know about you but when I’m stepping through code, sometimes I accidentally step into non-user code and It Is Very Annoying. Turns out, we can make it so that library code gets ignored using <code class="language-plaintext highlighter-rouge">gdb.Breakpoint</code>.</p>
<ul>
<li>Put commands that execute when a breakpoint is encountered</li>
<li>Use <code class="language-plaintext highlighter-rouge">libclang</code> to find out semantic info about the code</li>
</ul>
<h4 id="3-finding-cycles">3. Finding Cycles</h4>
<ul>
<li>Use valgrind, parse output of the monitor into a graph</li>
<li>Use graph processing to find loops in your program.</li>
</ul>
<h4 id="thoughts-5">Thoughts</h4>
<ul>
<li>I can try to take notes about things I constantly get annoyed at when debugging and try to come up with a solution. Perhaps, if it is a common enough problem, I can share it with others and they will find it useful as well.</li>
</ul>
<h2 id="my-lighting-talk">My Lighting Talk</h2>
<p>My lighting talk happened! During the day I took some time off to create my slides and @bunnyladame kindly looked over them after lunch. Since I did not have a lot of time to prepare, I couldn’t talk about anything too technical so I thought I would share a cool feature of git that I learned this summer while interning at Mozilla - <code class="language-plaintext highlighter-rouge">git worktree</code>. Git worktrees helped me speed up my development workflow because it allowed me to have more than one branch checked out simultaneously without cloning the repository twice. The rough idea is that you have two working trees in different directories, and they share <code class="language-plaintext highlighter-rouge">.git</code> folder and all of the stashes and other metadata. My slides are <a href="/assets/pdfs/Cppcon2018-Lighting-Talk.pdf">here</a>. And I will share the link to the video when it comes out!</p>
<p>Before my talk @hankadusikova also looked over my slides and gave me thumbs up.</p>
<p>People were really supportive - some of them took photos of me and/or tweeted that they learned a new thing, which made me really happy! Turns out there were quite a few people who have not heard of git worktrees before, so I’m glad I was able to share something new with people.</p>
<p>P.S. I used <a href="https://carbon.now.sh">carbon.sh</a> to produce beautiful ‘screenshots’ of my code.</p>
<h1 id="day-5---friday-sept-28-2018">Day 5 - Friday, Sept 28, 2018</h1>
<p>I was not feeling well in the first half of the day, so I was not able to take any notes. I’m definitely going to rewatch Mat Godbolt’s “The Bits Between the Bits: How We Get to main()” talk.</p>
<h1 id="summary">Summary</h1>
<h2 id="reflection-1">Reflection</h2>
<p>This part of the blog post is the hardest to write. I have a lot of Good Feelings about CppCon and I want to convey all of them. This has been one of the best experiences I have ever had. CppCon had a great atmosphere, and it was so easy to talk to people in the hallway. There was not a single moment in the conference when I felt alone or left out.</p>
<p>The talks were great and I come away with lots of things I want to learn more about. I’m feeling very inspired and motivated to explore new areas with C++ I have not explored before. There are some talks which have immediate impact on my work, like Jeff Trull’s talk on GDB Python APIs, and there are talks which will affect my work and thinking in the long run; to name a few - Kate Gregory’s “What Do We Mean When We Say Nothing At All?” and Richard Powell’s “Named Arguments From Scratch in C++”. There are also “C++ Community Building BoF” session and #include_cpp dinner that taught me things I would have been unable to read about in the books.</p>
<p>I have also learned lots of things about C++ not only through talks, but through informal interactions and discussions with people in the hallways, over lunches and dinners. Someone at the conference told me they thought CppCon was life-changing for them, and I wholeheartedly agree. I have met such fantastic and kind people (a lot of them from #include_cpp) that it makes me incredibly sad that CppCon has an end to it. Everyone I hanged out with was so supportive, encouraging and wholesome. They tried to introduce me to new people, explained things patiently when I didn’t know something and taught me new things about how to be a better human being. I feel incredibly lucky to have met all these wonderful people. I am graduating soon and as I begin the next stage of my life I’m happy to know that I am a part of such a great supportive community of people who share my interest in C++ and other things.</p>
<h2 id="other-random-good-things-that-happened-to-me-in-no-particular-order">Other Random Good Things That Happened To Me In No Particular Order</h2>
<ul>
<li>Sunny weather in Bellevue</li>
<li>Someone told me if I wanted to submit a proposal, they could help me edit/write the paper</li>
<li>Sushi buffet</li>
<li>Dumplings at Din Tai Fung</li>
</ul>
<h2 id="thank-you">Thank You</h2>
<p>I’m going to echo some of the things I said on my Twitter. I’m so grateful to Women in Tech Fund, #include_cpp and 43 people who donated to gofundme for giving me the opportunity to attend CppCon. I also want to thank all the organizators and volunteers of CppCon! And I want to thank everyone who has been kind to me - I hope we cross paths at CppCons in future.</p>Anny GCppCon2018 ended two weeks ago and I have been unable to write about it until now. A week away from school required me to catch up on a lot of material and assignments once I got back and that’s why it took me so long to write this. While I am still not free of school stress, at least I have time to describe how I ended up at CppCon 2018, how it was and what I learned. Spoiler alert: it was life-changing! :) Outline This post is very long, so here is an outline of Things. If you don’t have much time, I suggest you read the summary. I also have handwritten (sketch?)notes here. And here is a preview of some of my notes How I heard about CppCon Registering for CppCon 2018 About Women In Tech Fund and #include_cpp The weekend before CppCon Day 1 - Monday, Sept 24, 2018 Bjarne Stroustrup - Concepts: The Future of Generic Programming Matthew Butler - Secure Coding Best Practices: Your First Line Is The Last Line Of Defense (1/2) Lunch Simon Brand - How C++ Debuggers Work Student Dinner Grill the Committee Day 2 - Tuesday, Sept 25, 2018 Kate Gregory - What Do We Mean When We Say Nothing At All? Greg Law - More gdb and other Linux debugging wizardry Richard Powell - Named Arguments #include_cpp dinner Day 3 - Wednesday, Sept 26, 2018 Patricia Aas - Software Vulnerabilities Kate Gregory - Simplicity C++ Community Building BoF Serge Guelton - C++ in Elvenland Lighting Talks Day 4 - Thursday, Sept 27, 2018 Herb Sutter - Thoughts on a More Powerful And Simpler C++ Jeff Trull - Liberating the Debugging Experience with the GDB Python API My Lighting Talk Day 5 - Friday, Sept 28, 2018 Summary Reflection Other Random Good Things That Happened To Me In No Particular Order Thank You How I heard about CppCon C++ is one of my favourite programming languages and I like anything to do with C++. Around two years ago I discovered CppCon conference videos online, and have watched quite a few talks since then. Then I thought it must be so cool to attend this conference, watch speakers present different things one can do in C++ and meet people who are interested in the same things as you. Registering for CppCon 2018 This August, I was looking to see which conferences were happening nearby me and to my surprise I have realized that CppCon has always been in Bellevue (I am in Vancouver, Canada) and it was going to be held quite soon! Then I discovered that Women In Tech Fund was accepting applications from women who are in need of financial assistance to be able to attend the conference. I wrote about how I would benefit from attending such conference and what I hope to get out of it and submitted my application. The next day I have received a reply saying that my application has been accepted. Wow! It was a really happy day for me. After some time I was contacted by the organizers of the CppCon and they helped me register for CppCon and ensured that I had a place to stay for the duration of the conference. About Women In Tech Fund and #include_cpp I would like to share some information about @WomenInTechFund and @include_cpp because I think those are awesome organizations/communities that everyone should know about! First, a little bit about Independent Fund for Women in Tech (@WomenInTechFund) The Independent Women in Tech Fund aims to help women attend security conferences by providing assistance with entry ticket and possible travel support. […] For most women, the opportunity to attend conferences, talk to peers, see other women in the field, ask for advices and feel that they are supported makes a difference in their careers. Such support may help decide their future. We want to help with that. And a little bit about #include_cpp #include<C++> is a global, inclusive, and diverse community for developers interested in C++. Here, you can find a welcoming space to learn and discuss C++. We also provide resources to create safer, more inclusive, community gatherings. We believe that a community is only as good as how it treats its most vulnerable members. Therefore, we strive to create a welcoming, safe, and accessible environment. You can find our code of conduct here. So what’s going on? :) What happened and how did the two organizations work together to help women (cis and trans) to attend CppCon? To summarize the story from the gofundme campaign - #include_cpp has been in contact with the Women in Tech Fund and CppCon organizers, and were raising money for women to attend this conference. And Women In Tech Fund was administering the tickets. 43 people have donated to this cause! That is super awesome! The weekend before CppCon Reaching out to people on Twitter I wanted to do a shout out to Women In Tech Fund, #include_cpp and the 43 donors, so I thanked them on Twitter (since all the cool people are hanging out on Twitter) and then my tweet got a lot of retweets! People were very nice and told me they were glad I get a chance to attend CppCon they wanted to meet me and I should find them when I have a chance they would gladly introduce me to other people I also reached out myself to a few other people who were tweeting under #IGotYourBack, and I tweeted to them thanking them and saying that I hope I get to meet them at the conference. Deciding which talks to attend Now that I was going to CppCon for sure, I needed to decide which talks I should attend. It was. Really. Overwhelming. I wrote about it here. To summarize - I went through the description of each talk carefully, and wrote down what I know already in relation to the topic and what I hope to learn about it. I also tweeted about it and people liked it as well! Day 0 - Sunday, Sept 23, 2018 Cpp Tee Shirt Dinner I call it day 0 because the official talks did not begin until the next day. My plan for the day was to bus to Bellevue from Vancouver, to attend a Cpp Tee Shirt Dinner and to attend a reception. From what I understand, this dinner is sort of like a tradition, and you are supposed to wear a Cpp tshirt and go to one of the pre-selected restaraunts, alone or with a group. This gives people a chance to mingle and meet new people. Since I didn’t know anybody, I was really excited to attend the dinner. Also, on twitter, CppCon folks were sharing the names of the restaunts they were planning to head to, which was helpful. In the morning @pati_gallardo reached out to me, which was very very nice of her, and I decided I would go to the same restaraunt as her so I could meet her in person! I went there and met her and quite a few other people (spoiler alert # 1: they were super awesome people) and I had such a great time. I got to know some folks, where they work, what they work on and etc. We sat there for two hours or so and more and more people were joining, which was awesome! Reception After the dinner there was a reception back in the convention center and we went there. It was Patricia’s first time at the CppCon, but she already knew quite a few people and she made sure to introduce me to everyone she knew and anyone else that I wanted to meet. That. Was. Super. Fun! Everyone was so nice and interesting to talk to! Quite a few of them were folks I briefly interacted on Twitter. I had good conversations with people, learned new things (e.g. monads in C++) and met some people whose work I have seen before - e.g. Rob from @cppcast!!! what!! I used to listen to that podcast all the time while commuting to university. I also chatted with @hankadusikova and @michaelcaisse and they convinced me to do a lighting talk (spoiler alert #2: I did it)! Reflection I came back to my hotel and couldn’t stop smiling because I was really happy. I couldn’t believe how welcoming and nice everyone was. I got to participate in so many interesting conversations and no one acted condescending if I asked something that everyone else might have already known - quite the opposite - they were very eager to explain. Day 1 - Monday, Sept 24, 2018 Finally, I will talk about the talks I have seen. But first, a little disclaimer - I’m going to talk about what I took away from the talks. Sometimes the talks went over my head and I was only able to write down keywords and ideas I need to read up on. I think that’s okay! I also took lots of notes (with doodles of course) and I posted them on twitter and will be attaching them here as well. A lot of info that is contained in my notes will be repeated here because I’m looking over my notes and typing them up here into cohesive paragraphs. Bjarne Stroustrup - Concepts: The Future of Generic Programming This talk was not really oriented at technical language details. Bjarne said that individual features in isolation from the language are not the intent here. The goal of the talk was to improve generic programming so it becomes as simple as non-generic code, but to achieve that, we need to improve other features. One might try to find similarities between Types and Concepts. Roughly put, Types specify the set of operations for an object. With Concepts, however, you specify how one can use an object, and it is not just for functions or classes. Some advantages of concepts include: readability maintainability Great! Let’s look at some properties of Concepts: (Some) Properties of Concepts Conditional properties This allows you to express the following idea in the code “make it so -> operator can only be used if other is a class”. Hold on, can’t we use std::enable_if for this? Well no, they don’t really scale and lots of workarounds are needed. Reliability Sometimes auto is misused, like in the following example auto x = foo(1); // x is InputChannel The above code is not really maintainable and the comment contains a requirement that can’t be enforced by the compiler. Fear not, because with Concepts you can do this: InputChannel x = foo(1); + This is readable and can be enforced by the compiler. Concept of Concepts Concepts aren’t really a new thing! Think of compile time predicates - they are quite fundamental and we have had them for a while. Examples include developer’s head documentation code comments But now, we get to represent those ideas in code! To summarize, I will list (some) of the pros again: + Concepts are flexible and allow you to keep extending and working on them (because you probably won’t get it right the first time) + Once you use concepts you won’t go back because you begin thinking in terms of them + You express ideas more clearly so even a compiler can understand This change is not a detail, but a change in how we think, so it is a major change, and that’s is why it might take a while to get adopted. Thoughts I thought this talk was informative and now I understand the appeal and importance of Concepts. I also found this talk interesting because Bjarne stressed how this change is quite fundamental and it will change how we will think about it. Lunch I thought it was important to talk about how my lunch was, given that I was very worried about being alone at this conference. I joined the #include_cpp discord on Sunday and I thought I would ask on the #CppCon channel if anyone wanted to get lunch together. Folks from #include_cpp had a booth for first two days of the conference, and so it happened that we would gather there at lunch time and head to some food place. It was very nice and I enjoyed speaking to people in informal settings about what they work on and what they do, and to get to know more people from the community. I also thought it was very cool that at first it would be a small group of us waiting to go out for lunch and then 10 minutes later when we are actually heading out of the convention center, 5 more people have joined. Matthew Butler - Secure Coding Best Practices: Your First Line Is The Last Line Of Defense (1/2) So, unfortunately, I was only able to go to the first part of the talk, because during the second part of the talk I really wanted to attend Simon Brand’s talk on ‘How C++ Debuggers work’, but, I think I learned quite a few new things even from only one part of the talk. So here are my notes! Intro So some companies might think that they don’t really need to care much about security. Here are 3 Reasons Why A Company Thinks They Are Safe From Attackers presence of firewalls - unfortunately, it alone did not save other companies. code review and tests - code reviews are not focused on finding exploits we are too {small,big,etc} to be a target - if you are a big company, you have lots to lose, and if you are a small company, your security is (most likely) too weak. As a result, you are always a target. Terms It might be helpful to define what a critical system is. A critical system is any system, regardless of its priority, that any other systems can interact with. An example is a wireless printer. Matthew told a story from his life of a time when a printer in the lab(?) of someone he knew was used to infiltrate the system. Another term that needs to be defined is attack vector. Attack vectors are anything that one from outside can get into, e.g. command line interfaces. Techniques So, how do you make your programs safer? You need to remember to Validate incoming arguments Study the standard And remember warnings are errors. Listen to them! While these are not comprehensive, it might also help to know some of the things that Matthew Butler will look for when penetrating a system: copy/moves of memory not validating input data or verifying the sender use of open source libraries their weaknesses are your weaknesses unguarded internal interfaces don’t assume inside systems to be safe! complexity in code design Thoughts It was very interesting! There was also an explanation (or a live demo) of buffer overflows, and I always find those to be fun. I wrote down some things I should look up again, because I forgot the details of how they work ASLR - randomizing address space Stack canaries - putting known bit patterns that needs to be checked before returning from a function. The code can abort if the canary is incorrect. And all of this is enforced by the compiler! Return oriented programming Simon Brand - How C++ Debuggers Work ELF and DWARF ELF is a binary format for executable and it has headers and other useful things. There is also another format DWARF and it is a debug info format. It describes stuff about your program, e.g. what functions you have and what types they return, what types you have, etc. Debuggers usually consume DWARF, but sometimes ELF is enough for your debugger but usually for complicated things your debugger needs more info. Breakpoints So what are some things that come to your mind when you think of debuggers? I think of breakpoints. Turns out there are hardware breakpoints and software breakpoints. Hardware breakpoints have special registers, but the number of them is limited. They can break on read/write/execute of a certain address. In x86 there are only 4 registers and I think you have to write the breakpoint address into those registers. Software breakpoints are implemented in software. The running code is modified so that a breakpoint occurs. You can have an unlimited number of breakpoints. A limitation is that they can only break on execute. In x86 you have to replace the current instruction with int3. Other random bits and bobs These are some definitions and notes I took down regarding some concepts. They are a bit messy and have no story connecting them because some of this stuff went over my head :) mangling is cool! each function gets mangled differently and the process differs from compiler to compiler. process_vm_ready reads more than one page at a time?? DW-tag subprogram - each function in DWARF has entries for its beginning and end. This allows you to compare current PC to ranges of each function and discover the exact name of the current function that is being processed. DWARF gives you info about inlined functions. things related to **shared libraries ** r_debug link_map - tells you of all shared libraries r_brk__ - gets called when a library is being called Thoughts This was a super interesting talk( and topic) and my plan for when I have some more time is to read about all of this in greater detail. During the talk I had some questions that I’m sure I can answer once I start researching some more (or rewatch the talk) How is DWARF generated? What exactly happens when an instruction is replaced with int3? How do line breakpoints work? What’s the process of generating those? How does step out and step in work? Student Dinner For dinner, all of the full-time current students who were attending CppCon were invited to a dinner with other students and some famous people from C++ community (like Bjarne S.). I had a lot of fun! I met some other students and learned about the kinds of things they do with C++ and it was very interesting. Grill the Committee After the dinner there was ‘Grill the Committee’ session. Even though up until this point I haven’t really kept up with what is going on in the C++ community or different proposals, I learned about some ongoing concerns that people have or some areas of C++ that are of great interest/concern to many. Day 2 - Tuesday, Sept 25, 2018 Kate Gregory - What Do We Mean When We Say Nothing At All? This talk poached the reader to think about what the presence of certain keywords in the code communicates to the readers and how to ensure that lack thereof is also meaningful. This is a quote that I really liked from Kate - You should be writing your code as if your children will be maintaining it. In other words - ensure you are communicating everything you want with the code you are writing. Kate gave lots of advice on how to communicate your intentions via different constructs. [[fallthrough]] Using fallthrough does not make a difference at runtime, but the compiler will complain if you misuse it. on defaults Avoid defaults because not everyone knows it. Not using defaults allows you to communicate your intent better. Saves others time guessing if you have considered it For nothing to mean something, in other places you have to make your intent explicit on parameter passing const, & convey the meaning about your API design (examples in the talk) on traditional loops When you have a traditional loop, is it because it’s not touching some objects? If yes, well there are algorithms for that… There are lambdas too. Importance of communicating Clear code communicates with future readers Shows them why you did something (your intent) “What does it mean when you make such choices” Ensure your nothing speaks volumes. Thoughts I really like any talk that Kate gives. Communicating with future readers of your code is really important, and Kate gives lots of techniques for ensuring effective communication. Greg Law - More gdb and other Linux debugging wizardry Soo I got a bit confused during this talk so my notes are just bullet points of some bits of information I understood and things I will look up in future Intro There are two main classes of debugging tools: Checkers Does my code do a bad thing, X, Y, Z? Examples include valgrind, address sanitizers, etc. Debuggers Signals When the program receives a signal, it’s suspended and tracer(??) gets notified gdb might decide not to handle the signal handle SIGSYS stop print pass is how you can tell gdb to not handle certain signals Misc Valgrind is a platform on top of which other checkers, e.g. memcheck, are built. It also has a remote debug server. There are different kind of sanitizers - address, memory, leak, etc… Things to look up in detail ftrace - function tracer strace - traces all function calls of a process ltrace - dynamic calls of a process reverse-step in gdb?? live recorder in gdb?? I know that *trace are very useful tools to know about, but I just never happened to use them before. I think I used strace for something once, but that’s it. I definitely need to look up all of those tools to see what kinds of things are possible to use them for. Thoughts I am interested in the things presented in this talk, so when I have more time I will definitely rewatch this. Richard Powell - Named Arguments In this talk, Richard showed the process of turning an idea about C++ language extension into reality. This was one of my favorite live demos! Richard incrementally prototyped named arguments in C++ from scratch. At first, he started with posing the question - what if we wanted to go from foo(6) to foo(a=6) in C++? How would we start? Well, what if we had a map with arguments and their names? Then we could extract arguments from the map and feed them to the function. The high level steps are: construct the arguments extract the arguments unpack the arguments I really liked the way he structured the presentation and live demo. It was not confusing at all, even though I didn’t know hana boost library or some of the exact techniques that Richard used. At the end of the talk, Richard emphasized that it’s okay that this example is not production ready. The important thing that he wanted to demonstrate is how turn an idea into a product. Not everything can be production ready, but the important thing is journey and learning things. One of the things he left us with was “Find a question. Then ask yourself, how can you do this in/with C++? “ I found this talk to be very insightful on how to approach problems in C++. I learned a lot and I really liked that Richard emphasized the journey and learning things. Things to look up hana boost library hana literals What does header only library mean? Is the code only located in the header? Is that it? #include_cpp dinner Another cool thing happened - I got to go to a #include_cpp dinner. There were a lot of people from #include_cpp community. There was dinner and a panel discussion featuring prominent women from the C++ community. They discussed how they got started with C++, what challenges they faced and gave advice. The dinner was great - we ate good food and chatted with people at our table. I got to know some other women in C++ and what they work on. Then, the panel discussion began after dinner. I’m not sure how to describe it, but I teared up a lot and wanted to give everyone a hug. I’m so glad to have heard from all of those women. Hearing about their experiences made me aware that my experiences are not unique and it was helpful to hear their advice on dealing with various challenges they faced. I’m so grateful that they shared all of those things with us. I’m also very grateful that this dinner was organized, because it was definitely one of my favorite events during this week. Day 3 - Wednesday, Sept 26, 2018 Patricia Aas - Software Vulnerabilities Patricia’s talk focused on software vulnerabilities. Some of the lessons I learned Don’t depend on undefined behavior Sometimes compilers optimize away the memset Can be an issue if you are depending on it to erase your passwords!!! Shellcode - piece of code. Today it is meant to represent the fact that you are able to execute something natively Don’t take external strings as format strings. They might not supply enough args and printf will read whatever the next thing on stack is Patricia also showed how to perform a buffer overflow attack. I couldn’t help but notice how well organized the code examples were. I think everyone who wants to show code on their slides needs to follow Patricia’s examples. She highlighted different parts of the assembly or C code that we needed to look at, and it made it easier to follow along and not get distracted by other lines of code! Patricia also talked about X Things She Would Rather You Not Do to prevent certain vulnerabilities. I did not write them down because I was afraid I would not be able to explain them properly. So you will have to watch the talk :) Kate Gregory - Simplicity I got to introduce Kate for her keynote! Whaat? How did that happen? On Tuesday morning, Jon Kalb has emailed me and said that he really liked my blog post that Bryce forwarded to him. He asked if I would like to introduce Kate for her keynote. I couldn’t believe it! As I have previously mentioned, I am a huge fan of Kate, and I have watched a lot of her videos. So of course I said yes! Before I went on stage, Jon gave me some advice “Breathe. Smile. Speak slowly”, which I definitely tried to follow. It was such an honor to introduce Kate! :’) Here is a list of lessons learned (but of course, please exercise your judgement): When you teach, start simple. Omit things like error checking in order to lesson the cognitive burden. Remember, encapsulation != obscuration. Simpler code is harder to write! It is not often faster, because to make it faster you have to know somethign about the language. E.g. auto p vs auto p&. Short function names! If you can’t name it, it is probably several functions. No nesting, return from your function early. (Also, there was a lighting talk on Wednesday or Thursday called ‘Shape of a program’ by @JamesMcNellis dedicated to this topic alone. Highly entertaining and also useful.) Clean up in destructors. No one will forget to clean up, especially if they add a new throw statement somewhere in one of the functions. Think about the future. It might be easy now, but later it will confuse people. C++ Community Building BoF This was a Birds of a Feather session. The goal was to discuss how to build a successfull C++ community. I learned a lot of things and tried to write all of them down. Running meetups If you would like to organize a meetup, here are some ideas for how to do so: Reach out to speakers in other cities, in case they are travelling. If they are visiting your city, politely ask if they would like to speak at your event. It’s important not only to have talks but also discussions and have the meetup be a group and not just a service with talks. Have lighting talks What about organizing a book club for various C++ books/articles? Get people to showcase their projects. What about reviewing resumes and giving people some feedback? Sometimes have activities that allow everyone to break up into smaller groups Ensure diversity of topics. E.g. if you find yourself mostly presenting talks about debuggers, maybe think about presenting something about graphics in C++. Or add some non-technical talks. Sponsorship How should you talk to a company when you would like their sponsorship? Tell them what they will get in return e.g. you can provide them with 2 mins at the beginning of each talk, or distribute their swag, etc Tell them concrete things that you will need e.g. pizza and printed flyers Maybe look for sponsorships from companies who hire C++ developers, because they will have more incentive to help you Voluntering How do you get volunteers to help you with your meetup? Split up and rotate roles between people. Can assign different people to find speakers. Ask for specific things to get done e.g. we need someone to order pizza, receive the delivery and setup the pizzas You have to let go and not micro-manage people! Very important - appreciate volunteers - people are helping you in their free time, for free. Serge Guelton - C++ in Elvenland Another really cool talk I wish I could understand but it went over my head :( That’s okay though! I wrote down things I was confused about and I can read up on them later, and rewatch the talk. Symbol tables in ELF Symbol demangling Shrinking existing libraries UND in symbol table nm readelf objdump objcopy Lighting Talks Lighting talks are such a great idea. @michaelcaisse is a very talented and super hilarious host. One thing I have written down from the talks that I would like to look up is std::basic_string. Someone presented a lighting talk on using it instead of a vector. I’m intrigued. Day 4 - Thursday, Sept 27, 2018 Herb Sutter - Thoughts on a More Powerful And Simpler C++ This was a long talk, but my notes are succint because in the middle of it there was a demo. I really like the way the demo was done. It got the point across as one wants a demo to do, but without the panics that everything might crash and not work. I thought this talk was well presented, and even though I have not heard of meta classes before, Herb explained them well and I was able to understand their purpose and why they are a Good Thing. Below are some notes I took. When we add new things to the language, it is to Make user code simpler Remove edge cases, e.g. lifetimes Discourage certain things How do we know which features represent C++. Or to be more clear, what is C++? ** Don’t pay for what you don’t use (0 overhead). Of course, you still have to pay for it when you use it. Leave no room for lower language (except for assembly). Backwards compatibility What are properties of Good C++ Features? Express intent better Simplify C++ code (read, write, debug) Remove an exception to ** Remove gotchas Help us deprecate something (e.g. typedef -> using) Jeff Trull - Liberating the Debugging Experience with the GDB Python API This was one of my favorite talks!I was so blown away by this talk. I have previously heard of GDB’s python APIs but I never got a chance to play around with them. The documentation was kind of scary to me and I wasn’t really sure what is possible. Jeff’s presentation was great! His live demos were well prepared and they were truly necessary in his talk. This talk not only taught us about Python APIs, but also taught us how to approach any obstacles in gdb and what is possible. I learned a lot of things and as soon as I have some time, I will be playing around with those APIs. I can’t wait to build python hooks to make my debugging easier. Here are 3 Very Cool Things You Can Do With GDB Python APIs. 1. Improve Stack Traces Frame decorators are filters and you can override them to add custom behavior. Jeff showed us how we could replace verbose std names with well known values. 2. Easily Step To User Code I don’t know about you but when I’m stepping through code, sometimes I accidentally step into non-user code and It Is Very Annoying. Turns out, we can make it so that library code gets ignored using gdb.Breakpoint. Put commands that execute when a breakpoint is encountered Use libclang to find out semantic info about the code 3. Finding Cycles Use valgrind, parse output of the monitor into a graph Use graph processing to find loops in your program. Thoughts I can try to take notes about things I constantly get annoyed at when debugging and try to come up with a solution. Perhaps, if it is a common enough problem, I can share it with others and they will find it useful as well. My Lighting Talk My lighting talk happened! During the day I took some time off to create my slides and @bunnyladame kindly looked over them after lunch. Since I did not have a lot of time to prepare, I couldn’t talk about anything too technical so I thought I would share a cool feature of git that I learned this summer while interning at Mozilla - git worktree. Git worktrees helped me speed up my development workflow because it allowed me to have more than one branch checked out simultaneously without cloning the repository twice. The rough idea is that you have two working trees in different directories, and they share .git folder and all of the stashes and other metadata. My slides are here. And I will share the link to the video when it comes out! Before my talk @hankadusikova also looked over my slides and gave me thumbs up. People were really supportive - some of them took photos of me and/or tweeted that they learned a new thing, which made me really happy! Turns out there were quite a few people who have not heard of git worktrees before, so I’m glad I was able to share something new with people. P.S. I used carbon.sh to produce beautiful ‘screenshots’ of my code. Day 5 - Friday, Sept 28, 2018 I was not feeling well in the first half of the day, so I was not able to take any notes. I’m definitely going to rewatch Mat Godbolt’s “The Bits Between the Bits: How We Get to main()” talk. Summary Reflection This part of the blog post is the hardest to write. I have a lot of Good Feelings about CppCon and I want to convey all of them. This has been one of the best experiences I have ever had. CppCon had a great atmosphere, and it was so easy to talk to people in the hallway. There was not a single moment in the conference when I felt alone or left out. The talks were great and I come away with lots of things I want to learn more about. I’m feeling very inspired and motivated to explore new areas with C++ I have not explored before. There are some talks which have immediate impact on my work, like Jeff Trull’s talk on GDB Python APIs, and there are talks which will affect my work and thinking in the long run; to name a few - Kate Gregory’s “What Do We Mean When We Say Nothing At All?” and Richard Powell’s “Named Arguments From Scratch in C++”. There are also “C++ Community Building BoF” session and #include_cpp dinner that taught me things I would have been unable to read about in the books. I have also learned lots of things about C++ not only through talks, but through informal interactions and discussions with people in the hallways, over lunches and dinners. Someone at the conference told me they thought CppCon was life-changing for them, and I wholeheartedly agree. I have met such fantastic and kind people (a lot of them from #include_cpp) that it makes me incredibly sad that CppCon has an end to it. Everyone I hanged out with was so supportive, encouraging and wholesome. They tried to introduce me to new people, explained things patiently when I didn’t know something and taught me new things about how to be a better human being. I feel incredibly lucky to have met all these wonderful people. I am graduating soon and as I begin the next stage of my life I’m happy to know that I am a part of such a great supportive community of people who share my interest in C++ and other things. Other Random Good Things That Happened To Me In No Particular Order Sunny weather in Bellevue Someone told me if I wanted to submit a proposal, they could help me edit/write the paper Sushi buffet Dumplings at Din Tai Fung Thank You I’m going to echo some of the things I said on my Twitter. I’m so grateful to Women in Tech Fund, #include_cpp and 43 people who donated to gofundme for giving me the opportunity to attend CppCon. I also want to thank all the organizators and volunteers of CppCon! And I want to thank everyone who has been kind to me - I hope we cross paths at CppCons in future.Pre-reading about and preparing for attending Cppcon2018 talks.2018-09-22T00:00:00+00:002018-09-22T00:00:00+00:00https://annygakh.github.io/longblog/2018/09/22/prereadingCppcon2018talks<p>I needed to prepare for Cppcon and I decided to look at each talk and write down any thoughts I have for the ones I want to attend.</p>
<h2 id="monday-talks">Monday talks</h2>
<h4 id="900-am---concepts-the-future-of-generic-programming-by-bjarne-stroustrup">9:00 AM - Concepts: The Future of Generic Programming by Bjarne Stroustrup</h4>
<h4 id="1100-am---the-c-execution-model-by-bryce-adelstein-lelbach-see-in-person">11:00 AM - The C++ Execution Model by Bryce Adelstein Lelbach (see in person)</h4>
<p>I took Introduction to Computer Systems and Advanced Operating Systems courses at my university, where we studied how threads work - their interaction with each other, layout in memory and scheduling thereof by the OS - all in C. I’m not sure how different it will be in C++, but I’m excited for this talk anyway!</p>
<h4 id="1100-am---how-to-teach-c-and-influence-a-generation-by--christopher-di-bella-watch-online">11:00 AM - How to Teach C++ and Influence a Generation by Christopher Di Bella (watch online)</h4>
<p>This talk is happening at the same time as the one above, and, unfortunately, I don’t think I will be able to attend this one. But I’m excited to watch it afterwards. This talk resonates with me because I consume a lot of educational resources - from books to online tutorials - and not all of them are of the same quality and it will be interesting to see what the speaker proposes to achieve teaching C++ in an effective manner.</p>
<h4 id="200-pm-secure-coding-best-practices-your-first-line-is-the-last-line-of-defense-by-matthew-butler-see-in-person">2:00 PM Secure Coding Best Practices: Your First Line Is The Last Line Of Defense by Matthew Butler (see in person)</h4>
<p>From the talk description itself, this is what will be discussed</p>
<blockquote>
<ul>
<li>How hackers think and how they identify weaknesses in our systems.</li>
<li>How to identify hidden attack surfaces, attack vectors and vulnerabilities in critical systems.</li>
<li>Where the most common vulnerabilities in Modern and classic C++ are and how to avoid them.</li>
<li>Why avoiding parts of the language doesn’t help.</li>
<li>Where we can trade off security for performance.</li>
</ul>
</blockquote>
<p>This all sounds so intriguing! There will even be a demo of a classic buffer overflow exploit. Long time ago I tried to read “Hacking: The Art of Exploitation” by Jon Erickson, but I gave up at the very beginning. Perhaps, this talk will inspire me to revisit the book. :)</p>
<h4 id="315-pm---how-c-debuggers-work-by-simon-brand-see-in-person">3:15 PM - How C++ Debuggers Work by Simon Brand (see in person)</h4>
<p>Ahhhh!! I’m so excited for this talk! I don’t know much about debuggers - I frequently use gdb or lldb for debugging my programs, but I always wondered how it works. A few weeks ago I even added ‘Learn how debuggers work’ to my list of goals (I keep track of things I would like to learn when I have more time) even before knowing about this talk.</p>
<p>Simon even has a tutorial series <a href="https://blog.tartanllama.xyz/writing-a-linux-debugger-setup/">here</a> describing how to write a Linux debugger. After skimming his blog, I found <a href="https://blog.tartanllama.xyz/learning-cpp/">another post</a> I need to give a thorough read. I definitely will come back to look at more of his posts.</p>
<h4 id="445-pm---unwinding-the-stack-exploring-how-c-exceptions-work-on-windows-by-james-mcnellis-see-in-person">4:45 PM - Unwinding the Stack: Exploring How C++ Exceptions Work on Windows by James McNellis (see in person)</h4>
<p>Even though I am not a Windows developer, I am still very curious to learn about the low level details of how C++ exceptions work.</p>
<h4 id="315-pm---class-template-argument-deduction-in-c-17-by-timur-doumler-watch-online">3:15 PM - Class template argument deduction in C++ 17 by Timur Doumler (watch online)</h4>
<p>I know nothing about CATD but very eager to learn more. `nuff said :)</p>
<h4 id="not-a-talk---grill-the-committee">Not a talk - ‘Grill the Committee’</h4>
<p>This will be a panel discussion with the leaders of the C++ standards committee who will be taking questions from the audience.</p>
<h2 id="tuesday-talks">Tuesday talks</h2>
<h4 id="800-am---minidumps-gdb-compatible-software-controlled-core-dumps-by-matthew-fleming">8:00 AM - minidumps: gdb-compatible, software controlled core dumps by Matthew Fleming</h4>
<p>core dumps and gdb sounds like a lot of fun!</p>
<h4 id="900-pm---what-do-we-mean-when-we-say-nothing-at-all-by-kate-gregory">9:00 PM - What Do We Mean When We Say Nothing At All by Kate Gregory</h4>
<p>I have watched other Kate’s talks in the past and I really enjoyed them. Her talks always give me food for thought and I find them very interesting.</p>
<h4 id="1230-pm---debuggers-for-modern-applications-performance-and-static-analysis-by-samy-al-bahra">12:30 PM - Debuggers for Modern Applications: Performance and Static Analysis by Samy Al Bahra</h4>
<p>Debuggers. `nuff said.</p>
<h4 id="200-pm---more-gdb-and-other-linux-debugging-wizardry-by-greg-law">2:00 PM - More gdb and other Linux debugging wizardry by Greg Law</h4>
<p>Even without looking at the description I was already sold on this talk! From the talk description</p>
<blockquote>
<p>This talk will demonstrate some of the power of newer versions of GDB (Reverse debug, dynamic printf, amazing scriptability possibilities through Python), as well as some of the other Linux debugging tools at your disposal: ftrace, strace, ltrace, valgrind, rr, asan, and lots of very useful stuff in /proc.</p>
</blockquote>
<p>ftrace! strace! ltrace!! Ahhhhh, sounds amazing!</p>
<h4 id="315-pm---rvo-is-harder-than-it-looks-the-story-of--wreturn-std-move-by-arthur-odwyer">3:15 PM - RVO is Harder than it Looks: the story of -Wreturn-std-move by Arthur O’Dwyer</h4>
<p>I’m excited to hear about how Arthur has created a diagnostic for the Clang compiler and how one can contribute to Clang.</p>
<h4 id="350-pm---memory-tagging-and-how-it-improves-c-memory-safety-by-kostya-serebryany">3:50 PM - Memory Tagging and how it improves C++ memory safety by Kostya Serebryany</h4>
<p>Recently I have read <code class="language-plaintext highlighter-rouge">Delta Pointers: Buffer Overflow Checks Without the Checks</code> from EuroSys18 and it was my first time coming across pointer tagging. I wonder how different memory tagging is (I have not looked too much into the details).</p>
<h4 id="445-pm---named-arguments-in-c-from-scratch-by-richard-powell">4:45 PM - Named Arguments in C++ from Scratch by Richard Powell</h4>
<p>From my understanding of the description, we will learn how to extend the language using Hana, a library for metaprogramming, to allow support for named arguments. Very cool, indeed!</p>
<h4 id="830-pm---lighting-talks">8:30 PM - Lighting Talks</h4>
<p>Not sure what they will be about, but excited nevertheless!</p>
<h2 id="wednesday-talks">Wednesday talks</h2>
<p>Are you still reading this?</p>
<h4 id="900-am---fast-conversion-from-utf-8-with-c-dfas-and-sse-intrinsics-by-bob-steagall">9:00 AM - Fast Conversion From UTF-8 with C++, DFAs and SSE Intrinsics by Bob Steagall</h4>
<p>I know very little about UTF-8 and UTF-16, so I am hoping to learn more about this.</p>
<h4 id="1030-am---simplicity-not-just-for-beginners-by-kate-gregory">10:30 AM - Simplicity: not just for beginners by Kate Gregory</h4>
<p>Kate will be talking about how to make your C+ + code simpler, something that everyone needs to lear</p>
<h4 id="1230-pm---mixing-managed-and-unmanaged-code-and-targeting-cross-platform-distributions">12:30 PM - Mixing Managed and Unmanaged Code and Targeting Cross Platform Distributions</h4>
<blockquote>
<p>Even though we love C++, we don’t live in a C++-only world. In this session, you will be given the opportunity to ask questions about the best way to build applications in C++, yet also make them accessible to clients who use managed code. There will also be experts to answer questions about best practices for building C++ so it may be utilized efficiently in a cross-platform environment.</p>
</blockquote>
<h4 id="200-pm---understanding-optimizers-helping-the-compiler-help-you-by-nir-friedman">2:00 PM - Understanding Optimizers: Helping the Compiler Help You by Nir Friedman</h4>
<p>Hoping to learn more about optimizations done by the compiler.</p>
<h4 id="445-pm---c-in-elvenland-by-serge-guelton">4:45 PM - C++ in Elvenland by Serge Guelton</h4>
<p>Serge will be talking about binary format ELF and how understanding it can help a C++ developer understand the whole compilation chain.</p>
<h4 id="830-pm---optimizing-code-speed-and-spaswithchesce-with-build-time-switches">8:30 PM - Optimizing Code Speed and SpaSwithchesce with build Time Switches</h4>
<p>Compiler experts will be present to answer questions and engage in a discussion about compilers and linkers.</p>
<h2 id="thursday-talks">Thursday talks</h2>
<h4 id="900-am---oop-is-dead-long-live-data-oriented-design-by-stoyan-nikolov">9:00 AM - OOP is Dead, Long Live Data-oriented Design by Stoyan Nikolov</h4>
<p>Data oriented design will be introduced, and from my understanding, there will be a case study using Chrome browser.</p>
<h4 id="1030-am---thoughts-on-a-more-powerful-and-simpler-c-by-herb-sutter">10:30 AM - Thoughts on a More Powerful and Simpler C++ by Herb Sutter</h4>
<p>It is 10:30 PM at the moment of writing this, and I might be too tired, but I can’t summarize what this talk will be about. But! It seems interesting and I’m looking forward to it.</p>
<h4 id="200-pm---building-a-c-reflection-system-in-one-weekend-using-clang-and-llvm-by-arvid-gerstmann">2:00 PM - Building a C++ Reflection System in One Weekend Using Clang and LLVM by Arvid Gerstmann</h4>
<blockquote>
<p>In this talk we’ll go over the design and implementation of a
runtime reflection system, demonstrating the use of Clang and the
LLVM framework to craft custom C++ tools for your own needs.</p>
</blockquote>
<p>Count me in!</p>
<h4 id="330-pm-liberating-the-debugging-experience-with-the-gdb-and-python-api-by-jeff-trull">3:30 PM Liberating the Debugging Experience with the GDB and Python API by Jeff Trull</h4>
<p>Debuggers are cool, and often they have python APIs, which is extra cool because you can build your own custom tools using those APIs. So exciting!!</p>
<h4 id="830-pm---lighting-talks-1">8:30 PM - Lighting Talks</h4>
<p>:)</p>
<h2 id="friday-talks">Friday talks</h2>
<h4 id="900-am---debug-c-without-running-by-anastasiia-kazakova">9:00 AM - Debug C++ Without Running by Anastasiia Kazakova</h4>
<p>Anastasia is from jetbrains, and I have previously heard an episode of Cppcast where they have interviewed her. I also use CLion a lot, so I’m very interested in attending this talk to learn more about their solutions to various problems they encounter while building their IDE.</p>
<h4 id="1030-am---the-bits-between-the-bits-how-we-get-to-main-by-matt-godbolt">10:30 AM - The Bits Between the Bits: How We Get to main() by Matt Godbolt</h4>
<p>Ahh!!!</p>
<blockquote>
<p>When you run your C++ code, have you ever considered how the linker, loader, operating system, C and C++ runtime all work so hard to get everything set up for you to start running your code in main()?</p>
</blockquote>
<p>Can’t wait for this! I am always so curious, and in my Introduction to Computer Systems course we might have done something like this with C, but I have forgotten it now, and it would be great to see what happens before main() in C++.</p>
<h4 id="130-pm---clangd-architecture-of-a-scalable-c-language-server-by-ilya-biryukov">1:30 PM - Clangd: architecture of a scalable C++ language server by Ilya Biryukov</h4>
<p>A former colleague of mine recommended that I attend this talk, so I think I will do so! Even though I have not heard of LSP before. The mention of C++ text editors piqued my interest, because I’m very interested in how IDEs and text editors like vim are designed, and a while ago, I tried to develop my own vim-like text editor, as an exercise to learn more of C++.</p>
<h4 id="245-spectre-secrets-side-channels-sandboxes-and-security-by-chandler-carruth">2:45 Spectre: Secrets, Side-Channels, Sandboxes and Security by Chandler Carruth</h4>
<p>This talk will discuss different kinds of attacks.</p>
<h4 id="445-closing-panel-spectre">4:45 Closing Panel: Spectre</h4>
<p>This panel will consist of several security experts.</p>
<h2 id="final-thoughts">Final thoughts</h2>
<p>Wow! I did not realize how much work it is to go through the entire schedule and write down my thoughts for each talk I would like to attend. I can’t believe how many talks there will be, and I am so excited! At the same time, I am also so pleased to see that there are at least 15/20 mins between almost all of the talks. I am also very excited to see that in the evenings, around 8 PM, there are regularly scheduled lighting talks. I hope to spend most of my evenings attending those.</p>Anny GI needed to prepare for Cppcon and I decided to look at each talk and write down any thoughts I have for the ones I want to attend. Monday talks 9:00 AM - Concepts: The Future of Generic Programming by Bjarne Stroustrup 11:00 AM - The C++ Execution Model by Bryce Adelstein Lelbach (see in person) I took Introduction to Computer Systems and Advanced Operating Systems courses at my university, where we studied how threads work - their interaction with each other, layout in memory and scheduling thereof by the OS - all in C. I’m not sure how different it will be in C++, but I’m excited for this talk anyway! 11:00 AM - How to Teach C++ and Influence a Generation by Christopher Di Bella (watch online) This talk is happening at the same time as the one above, and, unfortunately, I don’t think I will be able to attend this one. But I’m excited to watch it afterwards. This talk resonates with me because I consume a lot of educational resources - from books to online tutorials - and not all of them are of the same quality and it will be interesting to see what the speaker proposes to achieve teaching C++ in an effective manner. 2:00 PM Secure Coding Best Practices: Your First Line Is The Last Line Of Defense by Matthew Butler (see in person) From the talk description itself, this is what will be discussed How hackers think and how they identify weaknesses in our systems. How to identify hidden attack surfaces, attack vectors and vulnerabilities in critical systems. Where the most common vulnerabilities in Modern and classic C++ are and how to avoid them. Why avoiding parts of the language doesn’t help. Where we can trade off security for performance. This all sounds so intriguing! There will even be a demo of a classic buffer overflow exploit. Long time ago I tried to read “Hacking: The Art of Exploitation” by Jon Erickson, but I gave up at the very beginning. Perhaps, this talk will inspire me to revisit the book. :) 3:15 PM - How C++ Debuggers Work by Simon Brand (see in person) Ahhhh!! I’m so excited for this talk! I don’t know much about debuggers - I frequently use gdb or lldb for debugging my programs, but I always wondered how it works. A few weeks ago I even added ‘Learn how debuggers work’ to my list of goals (I keep track of things I would like to learn when I have more time) even before knowing about this talk. Simon even has a tutorial series here describing how to write a Linux debugger. After skimming his blog, I found another post I need to give a thorough read. I definitely will come back to look at more of his posts. 4:45 PM - Unwinding the Stack: Exploring How C++ Exceptions Work on Windows by James McNellis (see in person) Even though I am not a Windows developer, I am still very curious to learn about the low level details of how C++ exceptions work. 3:15 PM - Class template argument deduction in C++ 17 by Timur Doumler (watch online) I know nothing about CATD but very eager to learn more. `nuff said :) Not a talk - ‘Grill the Committee’ This will be a panel discussion with the leaders of the C++ standards committee who will be taking questions from the audience. Tuesday talks 8:00 AM - minidumps: gdb-compatible, software controlled core dumps by Matthew Fleming core dumps and gdb sounds like a lot of fun! 9:00 PM - What Do We Mean When We Say Nothing At All by Kate Gregory I have watched other Kate’s talks in the past and I really enjoyed them. Her talks always give me food for thought and I find them very interesting. 12:30 PM - Debuggers for Modern Applications: Performance and Static Analysis by Samy Al Bahra Debuggers. `nuff said. 2:00 PM - More gdb and other Linux debugging wizardry by Greg Law Even without looking at the description I was already sold on this talk! From the talk description This talk will demonstrate some of the power of newer versions of GDB (Reverse debug, dynamic printf, amazing scriptability possibilities through Python), as well as some of the other Linux debugging tools at your disposal: ftrace, strace, ltrace, valgrind, rr, asan, and lots of very useful stuff in /proc. ftrace! strace! ltrace!! Ahhhhh, sounds amazing! 3:15 PM - RVO is Harder than it Looks: the story of -Wreturn-std-move by Arthur O’Dwyer I’m excited to hear about how Arthur has created a diagnostic for the Clang compiler and how one can contribute to Clang. 3:50 PM - Memory Tagging and how it improves C++ memory safety by Kostya Serebryany Recently I have read Delta Pointers: Buffer Overflow Checks Without the Checks from EuroSys18 and it was my first time coming across pointer tagging. I wonder how different memory tagging is (I have not looked too much into the details). 4:45 PM - Named Arguments in C++ from Scratch by Richard Powell From my understanding of the description, we will learn how to extend the language using Hana, a library for metaprogramming, to allow support for named arguments. Very cool, indeed! 8:30 PM - Lighting Talks Not sure what they will be about, but excited nevertheless! Wednesday talks Are you still reading this? 9:00 AM - Fast Conversion From UTF-8 with C++, DFAs and SSE Intrinsics by Bob Steagall I know very little about UTF-8 and UTF-16, so I am hoping to learn more about this. 10:30 AM - Simplicity: not just for beginners by Kate Gregory Kate will be talking about how to make your C+ + code simpler, something that everyone needs to lear 12:30 PM - Mixing Managed and Unmanaged Code and Targeting Cross Platform Distributions Even though we love C++, we don’t live in a C++-only world. In this session, you will be given the opportunity to ask questions about the best way to build applications in C++, yet also make them accessible to clients who use managed code. There will also be experts to answer questions about best practices for building C++ so it may be utilized efficiently in a cross-platform environment. 2:00 PM - Understanding Optimizers: Helping the Compiler Help You by Nir Friedman Hoping to learn more about optimizations done by the compiler. 4:45 PM - C++ in Elvenland by Serge Guelton Serge will be talking about binary format ELF and how understanding it can help a C++ developer understand the whole compilation chain. 8:30 PM - Optimizing Code Speed and SpaSwithchesce with build Time Switches Compiler experts will be present to answer questions and engage in a discussion about compilers and linkers. Thursday talks 9:00 AM - OOP is Dead, Long Live Data-oriented Design by Stoyan Nikolov Data oriented design will be introduced, and from my understanding, there will be a case study using Chrome browser. 10:30 AM - Thoughts on a More Powerful and Simpler C++ by Herb Sutter It is 10:30 PM at the moment of writing this, and I might be too tired, but I can’t summarize what this talk will be about. But! It seems interesting and I’m looking forward to it. 2:00 PM - Building a C++ Reflection System in One Weekend Using Clang and LLVM by Arvid Gerstmann In this talk we’ll go over the design and implementation of a runtime reflection system, demonstrating the use of Clang and the LLVM framework to craft custom C++ tools for your own needs. Count me in! 3:30 PM Liberating the Debugging Experience with the GDB and Python API by Jeff Trull Debuggers are cool, and often they have python APIs, which is extra cool because you can build your own custom tools using those APIs. So exciting!! 8:30 PM - Lighting Talks :) Friday talks 9:00 AM - Debug C++ Without Running by Anastasiia Kazakova Anastasia is from jetbrains, and I have previously heard an episode of Cppcast where they have interviewed her. I also use CLion a lot, so I’m very interested in attending this talk to learn more about their solutions to various problems they encounter while building their IDE. 10:30 AM - The Bits Between the Bits: How We Get to main() by Matt Godbolt Ahh!!! When you run your C++ code, have you ever considered how the linker, loader, operating system, C and C++ runtime all work so hard to get everything set up for you to start running your code in main()? Can’t wait for this! I am always so curious, and in my Introduction to Computer Systems course we might have done something like this with C, but I have forgotten it now, and it would be great to see what happens before main() in C++. 1:30 PM - Clangd: architecture of a scalable C++ language server by Ilya Biryukov A former colleague of mine recommended that I attend this talk, so I think I will do so! Even though I have not heard of LSP before. The mention of C++ text editors piqued my interest, because I’m very interested in how IDEs and text editors like vim are designed, and a while ago, I tried to develop my own vim-like text editor, as an exercise to learn more of C++. 2:45 Spectre: Secrets, Side-Channels, Sandboxes and Security by Chandler Carruth This talk will discuss different kinds of attacks. 4:45 Closing Panel: Spectre This panel will consist of several security experts. Final thoughts Wow! I did not realize how much work it is to go through the entire schedule and write down my thoughts for each talk I would like to attend. I can’t believe how many talks there will be, and I am so excited! At the same time, I am also so pleased to see that there are at least 15/20 mins between almost all of the talks. I am also very excited to see that in the evenings, around 8 PM, there are regularly scheduled lighting talks. I hope to spend most of my evenings attending those.CPSC 448 - Understanding TensorFlow2018-09-13T00:00:00+00:002018-09-13T00:00:00+00:00https://annygakh.github.io/longblog/2018/09/13/TensorFlow<p>The research project I am contributing to requires me to understand how TensorFlow works. Specifically, how distributed TensorFlow works. I have never worked with TensorFlow before, so at first, I need to understand how normal TensorFlow works, before understanding how it distributes tasks to more than one worker.</p>
<p>I have already taken a Machine Learning course in school, so I was able to get started with TensorFlow guides right away, without having to read up about it (which I will recommend for those who have not!).</p>
<h3 id="steps-to-understanding-tensorflow-with-the-focus-on-distributed-aspect">Steps to understanding TensorFlow with the focus on distributed aspect</h3>
<ol>
<li>TensorFlow’s Programming Model
<ul>
<li>Low level introduction to TensorFlow operations https://www.tensorflow.org/guide/low_level_intro</li>
<li>A lecture recording from Stanford. Watch until ~19min mark. https://youtu.be/PicxU81owCs?t=5m</li>
</ul>
</li>
<li>Distributed TensorFlow
<ul>
<li>A talk from TensorFlow Dev summit https://www.youtube.com/watch?v=la_M6bCV91M</li>
<li>An example program https://www.tensorflow.org/deploy/distributed. I would recommend reading this and understanding why different parts of the code are needed. However, this tutorial only provides you with a skeleton of the code, and not a complete program that is runnable out of the box, because you still need to write your own model. At this point, I still did not know how to create one, nor did I want to, because I just wanted to experiment deploying a TensorFlow cluster, without diving deep into creating models.</li>
<li>This is a distributed TensorFlow program that someone wrote https://github.com/ischlag/distributed-tensorflow-example. I followed this example to actually deploy a TensorFlow cluster, and see it in action.</li>
</ul>
</li>
</ol>
<p>Together, these resources were able to give me a high level overview of how TensorFlow works and how it distributes the tasks.</p>
<p>Below are some notes I took while watching Standford’s lecture on TensorFlow.</p>
<p><img src="/assets/images/IMG_6850.JPG" alt="Image" />
<img src="/assets/images/IMG_6851.JPG" alt="Image" /></p>Anny GThe research project I am contributing to requires me to understand how TensorFlow works. Specifically, how distributed TensorFlow works. I have never worked with TensorFlow before, so at first, I need to understand how normal TensorFlow works, before understanding how it distributes tasks to more than one worker. I have already taken a Machine Learning course in school, so I was able to get started with TensorFlow guides right away, without having to read up about it (which I will recommend for those who have not!). Steps to understanding TensorFlow with the focus on distributed aspect TensorFlow’s Programming Model Low level introduction to TensorFlow operations https://www.tensorflow.org/guide/low_level_intro A lecture recording from Stanford. Watch until ~19min mark. https://youtu.be/PicxU81owCs?t=5m Distributed TensorFlow A talk from TensorFlow Dev summit https://www.youtube.com/watch?v=la_M6bCV91M An example program https://www.tensorflow.org/deploy/distributed. I would recommend reading this and understanding why different parts of the code are needed. However, this tutorial only provides you with a skeleton of the code, and not a complete program that is runnable out of the box, because you still need to write your own model. At this point, I still did not know how to create one, nor did I want to, because I just wanted to experiment deploying a TensorFlow cluster, without diving deep into creating models. This is a distributed TensorFlow program that someone wrote https://github.com/ischlag/distributed-tensorflow-example. I followed this example to actually deploy a TensorFlow cluster, and see it in action. Together, these resources were able to give me a high level overview of how TensorFlow works and how it distributes the tasks. Below are some notes I took while watching Standford’s lecture on TensorFlow.Figuring out resolve hooks2018-08-08T00:00:00+00:002018-08-08T00:00:00+00:00https://annygakh.github.io/dailyblog/2018/08/08/ANoother<h1 id="links-that-seems-to-talk-about-resolve-hooks">Links that seems to talk about resolve hooks</h1>
<ul>
<li>https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JSPropertyOp</li>
<li>https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JSResolveOp</li>
<li>https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Internals/Property_cache</li>
<li>https://searchfox.org/mozilla-central/source/js/public/Class.h#473
<ul>
<li>This is the signature for the resolve operation</li>
</ul>
</li>
<li>https://searchfox.org/mozilla-central/source/js/src/builtin/Array.cpp#3763
<ul>
<li>Class definition that i could copy</li>
</ul>
</li>
<li>https://searchfox.org/mozilla-central/source/<strong>GENERATED</strong>/dom/bindings/WindowBinding.cpp#16761
<ul>
<li>examples of newEnumerate</li>
</ul>
</li>
</ul>Anny GLinks that seems to talk about resolve hooks https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JSPropertyOp https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JSResolveOp https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Internals/Property_cache https://searchfox.org/mozilla-central/source/js/public/Class.h#473 This is the signature for the resolve operation https://searchfox.org/mozilla-central/source/js/src/builtin/Array.cpp#3763 Class definition that i could copy https://searchfox.org/mozilla-central/source/GENERATED/dom/bindings/WindowBinding.cpp#16761 examples of newEnumerateMore work on code generator; Services.jsm;2018-07-30T00:00:00+00:002018-07-30T00:00:00+00:00https://annygakh.github.io/dailyblog/2018/07/30/12312321<h1 id="hiding-old-services-object-and-exposing-the-one-generated-via-template">Hiding old Services object and exposing the one generated via template</h1>
<p>So I have added all services from <code class="language-plaintext highlighter-rouge">Services.jsm</code> to <code class="language-plaintext highlighter-rouge">Services.py</code>
and firefox has compiled successfully, but one of the errors I get upon starting
Firefox is that <code class="language-plaintext highlighter-rouge">Services.search</code> is undefined. I define <code class="language-plaintext highlighter-rouge">Services.search</code> if MOZ_TOOLKIT_SEARCH is defined. But it seems as though MOZ_TOOLKIT_SEARCH is not defined, even though it’s needed. Let’s try to figure out how to define it.</p>
<p>I found <a href="https://searchfox.org/mozilla-central/rev/033d45ca70ff32acf04286244644d19308c359d5/toolkit/modules/moz.build#319-325">this code bit</a>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>for var in ('MOZ_TOOLKIT_SEARCH',
'MOZ_SYSTEM_NSS',
'MOZ_UPDATER',
'MOZ_ANDROID_MOZILLA_ONLINE',
'MOZ_SWITCHBOARD'):
if CONFIG[var]:
DEFINES[var] = True
</code></pre></div></div>
<p>So I added the following to <code class="language-plaintext highlighter-rouge">xpcom/build/moz.build</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>for var in ('MOZ_TOOLKIT_SEARCH',
'MOZ_CRASHREPORTER',
'MOZ_GECKO_PROFILER',
'ANDROID'):
if CONFIG[var]:
DEFINES[var] = True
</code></pre></div></div>
<p>because all of the above are mentioned in <code class="language-plaintext highlighter-rouge">Services.jsm</code>.</p>
<h1 id="servicesblocklistpluginqueries">Services.blocklist.pluginQueries</h1>
<p>I am running firefox, and while most of the things are working, I am getting the following error in the console:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>... TypeError: Services.blocklist.pluginQueries is undefined (resource://gre/modules/Blocklist.jsm:310:1) JS Stack trace: _init@Blocklist.jsm:310:1
</code></pre></div></div>
<p>So turns out <code class="language-plaintext highlighter-rouge">Blocklist</code> service, similar to <code class="language-plaintext highlighter-rouge">CrashManager</code>is also defined in JavaScript.
I need to figure out how to expose it to C++.</p>
<p>Next steps and observations</p>
<ul>
<li>Searching for <code class="language-plaintext highlighter-rouge">"@mozilla.org/extensions/blocklist;1"</code> in cpp files shows that when a service is retrieved using the above contract id, it is treated as <code class="language-plaintext highlighter-rouge">nsIBlocklistService</code>.</li>
<li>There is a header file for <code class="language-plaintext highlighter-rouge">nsIBlocklistService</code> class</li>
<li>So this service defined from JavaScript gets exposed to C++ somehow, so I should be able to figure out just from searching for <code class="language-plaintext highlighter-rouge">blocklist</code> or for <code class="language-plaintext highlighter-rouge">nsIBlocklistService</code></li>
<li>There is this <a href="https://searchfox.org/mozilla-central/rev/033d45ca70ff32acf04286244644d19308c359d5/toolkit/mozapps/extensions/extensions.manifest#7"><code class="language-plaintext highlighter-rouge">extensions.manifest</code></a> file, and I am not sure if I need this as well
<ul>
<li>I could check if there is <code class="language-plaintext highlighter-rouge">extensions.manifest</code></li>
</ul>
</li>
<li>Nevermind. I think what’s important here is that it seems that C++ code can use do_GetService to obtain <code class="language-plaintext highlighter-rouge">nsIBlocklistService</code>.</li>
<li>So it seems like we are obtaining Blocklist service correctly, but not all of the expected methods are exposed</li>
<li>Do we need to query for another interface perhaps? Let’s see. Need to search for <code class="language-plaintext highlighter-rouge">pluginQueries</code></li>
<li>Why is <code class="language-plaintext highlighter-rouge">Blocklist</code> defined in several places?
<ul>
<li>I looked at the blame of both files, and traced the creation of the second <code class="language-plaintext highlighter-rouge">Blocklist</code> definition back to this <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1456677">bug</a>.</li>
<li>So it seems like <code class="language-plaintext highlighter-rouge">Blocklist.jsm</code> is just a stub</li>
</ul>
</li>
<li>I think there are two problems at this point
<ol>
<li><code class="language-plaintext highlighter-rouge">Services.blocklist.pluginQueries</code> is not defined, and I think perhaps that due to
the field not being specified in the interface in the <code class="language-plaintext highlighter-rouge">idl</code> files</li>
<li><code class="language-plaintext highlighter-rouge">Services.blocklist.isLoaded</code> returns false when I comment out the old Service object and use mine, and returns true when the old Service object is used globally</li>
</ol>
</li>
<li>Let’s first figure out #2. <code class="language-plaintext highlighter-rouge">Cu.isModuleLoaded(BLOCKLIST_JSM)</code> returns false, so I guess we are not loading it properly when we are exposing it via C++.</li>
<li>Summary of the problem so far: Blocklist service is not defined purely in <code class="language-plaintext highlighter-rouge">Blocklist.jsm</code>. What we have in <code class="language-plaintext highlighter-rouge">Blocklist.jsm</code> is only a stub, and the real implementation is located in <code class="language-plaintext highlighter-rouge">toolkit/mozapps/extensions/addonManager.js</code>. The real implementation also contains some fields that are not exposed through <code class="language-plaintext highlighter-rouge">idl</code>. Should we go ahead and</li>
<li>Update, so my mentor said I should be using WrappedJsObject instead.</li>
</ul>
<h1 id="update-to-my-code-generator">Update to my code generator</h1>
<ul>
<li>Child process message manager does not need custom service retrieval and can be
retrieved in the same way as others, even though it is exposed via <code class="language-plaintext highlighter-rouge">webidl</code></li>
<li>Defines are available at build time, so already in the python script I can choose not to generate certain service getters</li>
<li>To be continued tomorrow…</li>
</ul>
<h1 id="useful-links">Useful links</h1>
<ul>
<li><a href="http://www.worldcolleges.info/sites/default/files/Creating_XPCOM_components.pdf">Creating XPCOM components pdf</a></li>
</ul>Anny GHiding old Services object and exposing the one generated via template So I have added all services from Services.jsm to Services.py and firefox has compiled successfully, but one of the errors I get upon starting Firefox is that Services.search is undefined. I define Services.search if MOZ_TOOLKIT_SEARCH is defined. But it seems as though MOZ_TOOLKIT_SEARCH is not defined, even though it’s needed. Let’s try to figure out how to define it. I found this code bit. for var in ('MOZ_TOOLKIT_SEARCH', 'MOZ_SYSTEM_NSS', 'MOZ_UPDATER', 'MOZ_ANDROID_MOZILLA_ONLINE', 'MOZ_SWITCHBOARD'): if CONFIG[var]: DEFINES[var] = True So I added the following to xpcom/build/moz.build for var in ('MOZ_TOOLKIT_SEARCH', 'MOZ_CRASHREPORTER', 'MOZ_GECKO_PROFILER', 'ANDROID'): if CONFIG[var]: DEFINES[var] = True because all of the above are mentioned in Services.jsm. Services.blocklist.pluginQueries I am running firefox, and while most of the things are working, I am getting the following error in the console: ... TypeError: Services.blocklist.pluginQueries is undefined (resource://gre/modules/Blocklist.jsm:310:1) JS Stack trace: _init@Blocklist.jsm:310:1 So turns out Blocklist service, similar to CrashManageris also defined in JavaScript. I need to figure out how to expose it to C++. Next steps and observations Searching for "@mozilla.org/extensions/blocklist;1" in cpp files shows that when a service is retrieved using the above contract id, it is treated as nsIBlocklistService. There is a header file for nsIBlocklistService class So this service defined from JavaScript gets exposed to C++ somehow, so I should be able to figure out just from searching for blocklist or for nsIBlocklistService There is this extensions.manifest file, and I am not sure if I need this as well I could check if there is extensions.manifest Nevermind. I think what’s important here is that it seems that C++ code can use do_GetService to obtain nsIBlocklistService. So it seems like we are obtaining Blocklist service correctly, but not all of the expected methods are exposed Do we need to query for another interface perhaps? Let’s see. Need to search for pluginQueries Why is Blocklist defined in several places? I looked at the blame of both files, and traced the creation of the second Blocklist definition back to this bug. So it seems like Blocklist.jsm is just a stub I think there are two problems at this point Services.blocklist.pluginQueries is not defined, and I think perhaps that due to the field not being specified in the interface in the idl files Services.blocklist.isLoaded returns false when I comment out the old Service object and use mine, and returns true when the old Service object is used globally Let’s first figure out #2. Cu.isModuleLoaded(BLOCKLIST_JSM) returns false, so I guess we are not loading it properly when we are exposing it via C++. Summary of the problem so far: Blocklist service is not defined purely in Blocklist.jsm. What we have in Blocklist.jsm is only a stub, and the real implementation is located in toolkit/mozapps/extensions/addonManager.js. The real implementation also contains some fields that are not exposed through idl. Should we go ahead and Update, so my mentor said I should be using WrappedJsObject instead. Update to my code generator Child process message manager does not need custom service retrieval and can be retrieved in the same way as others, even though it is exposed via webidl Defines are available at build time, so already in the python script I can choose not to generate certain service getters To be continued tomorrow… Useful links Creating XPCOM components pdfWorking on code generator2018-07-28T00:00:00+00:002018-07-28T00:00:00+00:00https://annygakh.github.io/dailyblog/2018/07/28/Components<h3 id="addressing-differences-in-how-components-are-defined">Addressing differences in how components are defined</h3>
<p><code class="language-plaintext highlighter-rouge">Services.jsm</code> exposes a bunch of components and not all of them are treated equally.
Some of them are exposed only if another class has been registered, or if the current
platform is android. Sometimes we want to expose additional interfaces that we know the component implements for sure and sometimes the component might not be implementing those interfaces. Trying to take care of all of those differences
and passing extra parameters to the <code class="language-plaintext highlighter-rouge">Service</code> object (in python) will be a nightmare and not very elegant.</p>
<p>For example,</p>
<ul>
<li>for <code class="language-plaintext highlighter-rouge">appinfo</code> component, it might be the case that it does not implement <code class="language-plaintext highlighter-rouge">nsIXULAppInfo</code> so we will need custom interface exposure for this component</li>
<li><code class="language-plaintext highlighter-rouge">crashmanager</code> component is implemented in JavaScript, so I will have to retrieve it in a different way</li>
<li>component with contract id <code class="language-plaintext highlighter-rouge">@mozilla.org/childprocessmessagemanager;1</code> is defined via webidl, so I will have to retrieve it in a different way as well</li>
<li><code class="language-plaintext highlighter-rouge">androidbridge</code> component is only exposed if the platform is android</li>
<li>component named <code class="language-plaintext highlighter-rouge">policies</code> is only exposed if <code class="language-plaintext highlighter-rouge">"@mozilla.org/browser/enterprisepolicies;1" in Cc</code></li>
</ul>
<p>So, at least</p>
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" />2 components need to be retrieved in a different way</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" />2 components might need to be exposed only under certain conditions</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" />1 component might not implement an interface we want</li>
</ul>
<p>So here is my plan</p>
<ul>
<li>Abstract out common template code that can be reused between most of the above scenarios</li>
<li>Modify <code class="language-plaintext highlighter-rouge">Service</code> class to take the following boolean flags
<ul>
<li><code class="language-plaintext highlighter-rouge">isCustomServiceGetter</code> to signify that component retrieval will be custom</li>
<li><code class="language-plaintext highlighter-rouge">isCustomInterfaceDefinition</code> to signify that querying for interfaces will be custom</li>
<li><code class="language-plaintext highlighter-rouge">isCustomExposure</code> to signify that the getter will be exposed conditionally
<ul>
<li>this will have to be inserted just before we define a property on the <code class="language-plaintext highlighter-rouge">Services</code> object</li>
</ul>
</li>
</ul>
</li>
<li>Allow users to specify custom template generation functions</li>
</ul>
<p>I was also thinking about how I will eliminate all calls to <code class="language-plaintext highlighter-rouge">Services.jsm</code> or at least make it so they are forwarded to <code class="language-plaintext highlighter-rouge">Components.Services</code>. For testing, I could make it so that I comment out all of <code class="language-plaintext highlighter-rouge">Services.jsm</code> and instead define a global like so
<code class="language-plaintext highlighter-rouge">const Services = Components.Services</code>. This will allow me to quickly test if all of the Services are working as expected instead of replacing all of them at once and discovering some of them are not working.</p>
<p>Another thing I could do is to create a mochitest of some sort and get each service manually and ensure that I get no errors while retrieving them. This can be an additional level of testing, in case not all of the exposed services are used by JavaScript consumers.</p>
<h3 id="componentsclasses">Components.classes</h3>
<p>I need to figure out what exactly the following code bit means</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="p">(</span><span class="dl">"</span><span class="s2">@mozilla.org/browser/enterprisepolicies;1</span><span class="dl">"</span> <span class="k">in</span> <span class="nx">Cc</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">initTable</span><span class="p">.</span><span class="nx">policies</span> <span class="o">=</span> <span class="p">[</span><span class="dl">"</span><span class="s2">@mozilla.org/browser/enterprisepolicies;1</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">nsIEnterprisePolicies</span><span class="dl">"</span><span class="p">];</span>
<span class="p">}</span>
</code></pre></div></div>
<p>From <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Language_Bindings/Components.classes">this mdn article</a></p>
<blockquote>
<p>Note that <code class="language-plaintext highlighter-rouge">Components.classes</code> reflects only those component classes that have been previously installed and registered with the component manager using <code class="language-plaintext highlighter-rouge">ContractID</code>s. If you want to use a class which was only registered with their CID, use <code class="language-plaintext highlighter-rouge">Components.classesByID</code> instead of <code class="language-plaintext highlighter-rouge">Components.classes</code> to retrieve it.</p>
</blockquote>
<p>How do we check if a certain component has registered with their contract id in C++?</p>
<p>Fast forward to 3 minutes later, somehow, I stumbled upon <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference">XPCOM Reference</a>, and from there I found <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Core_functions">Core XPCOM functions</a> and there I saw this:</p>
<blockquote>
<p>The <code class="language-plaintext highlighter-rouge">NS_GetComponentRegistrar</code> function returns a reference to the XPCOM Component Registrar.</p>
</blockquote>
<p>Registrar sounds like something that could be useful to me! Looking at the <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIComponentRegistrar">method overview of nsIComponentRegistrar</a> we can see that one of the functions is <code class="language-plaintext highlighter-rouge">boolean isContractIDRegistered(in string aContractID);</code>. From the documentation,</p>
<blockquote>
<p>This method is used to test for the existence of a class implementing a specific <code class="language-plaintext highlighter-rouge">ContractID</code>.</p>
</blockquote>
<p>I think this is what I was looking for! To double check, I will do a code search for the use of this function to see if it’s ever used in C++ code.</p>
<p>Indeed, it is used in several places including <a href="https://searchfox.org/mozilla-central/source/dom/webbrowserpersist/WebBrowserPersistLocalDocument.cpp#1223-1229">here</a>:</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">nsCOMPtr</span><span class="o"><</span><span class="n">nsIComponentRegistrar</span><span class="o">></span> <span class="n">registrar</span><span class="p">;</span>
<span class="n">nsresult</span> <span class="n">rv</span> <span class="o">=</span> <span class="n">NS_GetComponentRegistrar</span><span class="p">(</span><span class="n">getter_AddRefs</span><span class="p">(</span><span class="n">registrar</span><span class="p">));</span>
<span class="n">MOZ_ASSERT</span><span class="p">(</span><span class="n">NS_SUCCEEDED</span><span class="p">(</span><span class="n">rv</span><span class="p">));</span>
<span class="k">if</span> <span class="p">(</span><span class="n">NS_SUCCEEDED</span><span class="p">(</span><span class="n">rv</span><span class="p">)</span> <span class="o">&&</span> <span class="n">registrar</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">bool</span> <span class="n">result</span><span class="p">;</span>
<span class="n">rv</span> <span class="o">=</span> <span class="n">registrar</span><span class="o">-></span><span class="n">IsContractIDRegistered</span><span class="p">(</span><span class="n">contractID</span><span class="p">.</span><span class="n">get</span><span class="p">(),</span> <span class="o">&</span><span class="n">result</span><span class="p">);</span>
<span class="n">MOZ_ASSERT</span><span class="p">(</span><span class="n">NS_SUCCEEDED</span><span class="p">(</span><span class="n">rv</span><span class="p">));</span>
<span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>Anny GAddressing differences in how components are defined Services.jsm exposes a bunch of components and not all of them are treated equally. Some of them are exposed only if another class has been registered, or if the current platform is android. Sometimes we want to expose additional interfaces that we know the component implements for sure and sometimes the component might not be implementing those interfaces. Trying to take care of all of those differences and passing extra parameters to the Service object (in python) will be a nightmare and not very elegant. For example, for appinfo component, it might be the case that it does not implement nsIXULAppInfo so we will need custom interface exposure for this component crashmanager component is implemented in JavaScript, so I will have to retrieve it in a different way component with contract id @mozilla.org/childprocessmessagemanager;1 is defined via webidl, so I will have to retrieve it in a different way as well androidbridge component is only exposed if the platform is android component named policies is only exposed if "@mozilla.org/browser/enterprisepolicies;1" in Cc So, at least 2 components need to be retrieved in a different way 2 components might need to be exposed only under certain conditions 1 component might not implement an interface we want So here is my plan Abstract out common template code that can be reused between most of the above scenarios Modify Service class to take the following boolean flags isCustomServiceGetter to signify that component retrieval will be custom isCustomInterfaceDefinition to signify that querying for interfaces will be custom isCustomExposure to signify that the getter will be exposed conditionally this will have to be inserted just before we define a property on the Services object Allow users to specify custom template generation functions I was also thinking about how I will eliminate all calls to Services.jsm or at least make it so they are forwarded to Components.Services. For testing, I could make it so that I comment out all of Services.jsm and instead define a global like so const Services = Components.Services. This will allow me to quickly test if all of the Services are working as expected instead of replacing all of them at once and discovering some of them are not working. Another thing I could do is to create a mochitest of some sort and get each service manually and ensure that I get no errors while retrieving them. This can be an additional level of testing, in case not all of the exposed services are used by JavaScript consumers. Components.classes I need to figure out what exactly the following code bit means if ("@mozilla.org/browser/enterprisepolicies;1" in Cc) { initTable.policies = ["@mozilla.org/browser/enterprisepolicies;1", "nsIEnterprisePolicies"]; } From this mdn article Note that Components.classes reflects only those component classes that have been previously installed and registered with the component manager using ContractIDs. If you want to use a class which was only registered with their CID, use Components.classesByID instead of Components.classes to retrieve it. How do we check if a certain component has registered with their contract id in C++? Fast forward to 3 minutes later, somehow, I stumbled upon XPCOM Reference, and from there I found Core XPCOM functions and there I saw this: The NS_GetComponentRegistrar function returns a reference to the XPCOM Component Registrar. Registrar sounds like something that could be useful to me! Looking at the method overview of nsIComponentRegistrar we can see that one of the functions is boolean isContractIDRegistered(in string aContractID);. From the documentation, This method is used to test for the existence of a class implementing a specific ContractID. I think this is what I was looking for! To double check, I will do a code search for the use of this function to see if it’s ever used in C++ code. Indeed, it is used in several places including here: nsCOMPtr<nsIComponentRegistrar> registrar; nsresult rv = NS_GetComponentRegistrar(getter_AddRefs(registrar)); MOZ_ASSERT(NS_SUCCEEDED(rv)); if (NS_SUCCEEDED(rv) && registrar) { bool result; rv = registrar->IsContractIDRegistered(contractID.get(), &result); MOZ_ASSERT(NS_SUCCEEDED(rv)); ... }Jul, 26, 20182018-07-26T00:00:00+00:002018-07-26T00:00:00+00:00https://annygakh.github.io/dailyblog/2018/07/26/AnotherNewDay<p>I am realizing my blog posts don’t look very nice outside of my editor’s markdown preview,
so I will try to fix that soon. My apologies :)</p>
<h3 id="things-i-need-to-figure-out">Things I need to figure out</h3>
<ul>
<li>How to define lazy getters for services that implement more than one interface
<ul>
<li>Currently I am struggling with getting a ‘tear off’ of the object</li>
</ul>
</li>
<li>How to export a javascript service in an object</li>
</ul>Anny GI am realizing my blog posts don’t look very nice outside of my editor’s markdown preview, so I will try to fix that soon. My apologies :) Things I need to figure out How to define lazy getters for services that implement more than one interface Currently I am struggling with getting a ‘tear off’ of the object How to export a javascript service in an objectXPCOM, Components, Services, QueryInterface, Jul, 25, 20182018-07-24T00:00:00+00:002018-07-24T00:00:00+00:00https://annygakh.github.io/dailyblog/2018/07/24/NewDay<h3 id="some-of-the-things-mozilla-foundation-is-involved-with">Some of the things Mozilla Foundation is involved with</h3>
<ul>
<li>https://foundation.mozilla.org/campaigns/venmo-put-our-privacy-first/
<ul>
<li>Venmo has been making transaction details public by default and Mozilla wants to change that by raising awareness about it and urging Venmo to enable privacy settings by default</li>
</ul>
</li>
<li>https://publicbydefault.fyi/
<ul>
<li>Explains different insights one can get just by looking at the users’ public transactions.</li>
</ul>
</li>
<li>https://learning.mozilla.org/en-US/gigabit
<blockquote>
<p>The Mozilla Gigabit Community Fund provides grant funding in select U.S. communities to support pilot tests of gigabit technologies such as virtual reality, 4K video, artificial intelligence, and their related curricula. In so doing, our goal is to increase participation in technology innovation in support of a healthy Internet where all people are empowered, safe, and independent online.</p>
</blockquote>
</li>
</ul>
<h3 id="useful-links">Useful links</h3>
<ul>
<li>https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Guide/Creating_components</li>
<li>Learned a lot about components and classes, etc</li>
</ul>
<h3 id="wip">WIP</h3>
<p>I have managed to get the pipeline for the code generator to work for one simple service <code class="language-plaintext highlighter-rouge">scriptloader</code>. I now need to look into more services, and see how I can implement the complicated ones. The one I am currently looking at is <code class="language-plaintext highlighter-rouge">io</code>. In <code class="language-plaintext highlighter-rouge">services.jsm</code> there is
a following code</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">XPCOMUtils</span><span class="p">.</span><span class="nx">defineLazyGetter</span><span class="p">(</span><span class="nx">Services</span><span class="p">,</span> <span class="dl">"</span><span class="s2">io</span><span class="dl">"</span><span class="p">,</span> <span class="p">()</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">Cc</span><span class="p">[</span><span class="dl">"</span><span class="s2">@mozilla.org/network/io-service;1</span><span class="dl">"</span><span class="p">]</span>
<span class="p">.</span><span class="nx">getService</span><span class="p">(</span><span class="nx">Ci</span><span class="p">.</span><span class="nx">nsIIOService</span><span class="p">)</span>
<span class="p">.</span><span class="nx">QueryInterface</span><span class="p">(</span><span class="nx">Ci</span><span class="p">.</span><span class="nx">nsISpeculativeConnect</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>
<p>So when we call <code class="language-plaintext highlighter-rouge">Services.io</code> we will get the object that is returned by the above lambda function.</p>
<h3 id="componentsclasses">Components.classes</h3>
<p>According to this <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Language_Bindings/Components.classes">MDN</a> article,</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">Components.classes</code> is a read-only object whose properties implement the <code class="language-plaintext highlighter-rouge">nsIJSCID</code> interface. Each object represents one of the classes of XPCOM components that can be constructed or accessed as an XPCOM service.</p>
</blockquote>
<h4 id="query-interface">Query Interface</h4>
<p>Let’s recall <code class="language-plaintext highlighter-rouge">QueryInterface</code> function. From reading this <a href="https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XUL/Tutorial/XPCOM_Interfaces">MDN</a> article</p>
<blockquote>
<p>The function <code class="language-plaintext highlighter-rouge">QueryInterface(...)</code> is a function provided by all components which can be used to get a specific interface of that component. This function takes one parameter, the interface that you want to get.</p>
</blockquote>
<p>and from reading <a href="http://books.mozdev.org/html/mozilla-chp-8.html#mozilla-CHP-8-EX-1">Creating Applications with Mozilla </a> book,</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">QueryInterface</code></p>
<p>A method used to match an interface with a desired implementation. The implementation can be in C, C++, JavaScript, Python, and other languages for which appropriate bindings are created. You can have multiple implementations for an interface. Through <code class="language-plaintext highlighter-rouge">QueryInterface</code>, you can ask for and assign the desired interface to its implementation. Each XPCOM object needs to implement <code class="language-plaintext highlighter-rouge">QueryInterface</code> in order to return an instance of that object’s class.</p>
</blockquote>
<p>And also, tangentially related, I learned more about <code class="language-plaintext highlighter-rouge">Components</code> object. According to the above book,</p>
<blockquote>
<p>In XPConnect, XPCOM interfaces, <code class="language-plaintext highlighter-rouge">classIDs</code>, and <code class="language-plaintext highlighter-rouge">progIDs</code> are stored as global JavaScript objects and properties that can be manipulated directly through a top-level object called <code class="language-plaintext highlighter-rouge">Components</code>. This object accesses any component that is declared “scriptable” in an XPCOM IDL interface. Through the Components object, you can access and use the services that these interfaces provide. The Component object’s top-level properties and methods include: <code class="language-plaintext highlighter-rouge">QueryInterface</code>, <code class="language-plaintext highlighter-rouge">interfaces</code></p>
</blockquote>
<p>and more! You might have guessed it but here is what exactly <code class="language-plaintext highlighter-rouge">interfaces</code> are:</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">interfaces</code></p>
<p>A read-only object array containing all the interfaces declared scriptable in the IDL file. The object name has the same name as the interface it represents.</p>
</blockquote>
<h3 id="confusion">Confusion</h3>
<p>I have been reading a lot about XPCOM and I am forgetting the difference between components, classes, services.</p>
<p>Some of the functionality that the browser provides needs to be used by both consumers in core browser written in C++ and by the front-end of the browser which is written in JavaScript. XPCOM provides us with a way to implement such functionality in one language and expose it to a different language. There are clearly defined interfaces, which are language agnostic and they are specified using <code class="language-plaintext highlighter-rouge">IDL</code>, and they attempt to outline different functionality that a browser can provide. A <strong>component</strong> is a reusable piece of code that implements such an interface (one or more of such interfaces). Some of the components need to be instantiated every time they are used and they are called <strong>classes</strong>, and others exist as singleton instances and don’t need to be invoked every time they are used and they are called <strong>services</strong>.</p>
<h4 id="thoughts-in-progress">Thoughts In Progress</h4>
<p>So now we see that in the above code example, <code class="language-plaintext highlighter-rouge">Services.io</code> needs to return a <code class="language-plaintext highlighter-rouge">nsIIOService</code> service that also has an interface <code class="language-plaintext highlighter-rouge">nsISpeculativeConnect</code>. Let’s check if we can find a class that implements both such things, to see if I understood correctly everything I just read.</p>
<p>Indeed, <a href="https://searchfox.org/mozilla-central/rev/bdfd20ef30d521b57d5b6feeda71325e8b4cad66/netwerk/base/nsIOService.h#52">nsIOService</a> implements both. Now, I need to think about how I will make sure that the correct class will be returned when the getter is called. I’m going to look at the implementation of <code class="language-plaintext highlighter-rouge">QueryInterface</code>.</p>
<h4 id="update">Update</h4>
<p>My mentor pointed me towards this <a href="https://searchfox.org/mozilla-central/rev/bdfd20ef30d521b57d5b6feeda71325e8b4cad66/js/xpconnect/src/XPCJSID.cpp#498-499">code</a> so I will be doing something like this</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="p">(</span><span class="n">IS_WN_REFLECTOR</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span>
<span class="k">auto</span> <span class="n">tearOff</span> <span class="o">=</span> <span class="n">XPCWrappedNative</span><span class="o">::</span><span class="n">Get</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span><span class="o">-></span><span class="n">FindTearOff</span><span class="p">(</span><span class="o">*</span><span class="n">iid</span><span class="p">);</span>
</code></pre></div></div>
<p>to get a tear off object which will allow us to access both interfaces.</p>
<h4 id="more-updates">More updates</h4>
<p>I kept trying to include <code class="language-plaintext highlighter-rouge">xpcprivate.h</code> but the compiler kept giving me an error</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>... Services.cpp:31:10: fatal error: 'XPCWrappedNative.cpp' file not found
</code></pre></div></div>
<p>Then I realized I need to modify <code class="language-plaintext highlighter-rouge">xpcom/build/moz.build</code> so that it will include the appropriate files in the build.</p>Anny GSome of the things Mozilla Foundation is involved with https://foundation.mozilla.org/campaigns/venmo-put-our-privacy-first/ Venmo has been making transaction details public by default and Mozilla wants to change that by raising awareness about it and urging Venmo to enable privacy settings by default https://publicbydefault.fyi/ Explains different insights one can get just by looking at the users’ public transactions. https://learning.mozilla.org/en-US/gigabit The Mozilla Gigabit Community Fund provides grant funding in select U.S. communities to support pilot tests of gigabit technologies such as virtual reality, 4K video, artificial intelligence, and their related curricula. In so doing, our goal is to increase participation in technology innovation in support of a healthy Internet where all people are empowered, safe, and independent online. Useful links https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Guide/Creating_components Learned a lot about components and classes, etc WIP I have managed to get the pipeline for the code generator to work for one simple service scriptloader. I now need to look into more services, and see how I can implement the complicated ones. The one I am currently looking at is io. In services.jsm there is a following code XPCOMUtils.defineLazyGetter(Services, "io", () => { return Cc["@mozilla.org/network/io-service;1"] .getService(Ci.nsIIOService) .QueryInterface(Ci.nsISpeculativeConnect); }); So when we call Services.io we will get the object that is returned by the above lambda function. Components.classes According to this MDN article, Components.classes is a read-only object whose properties implement the nsIJSCID interface. Each object represents one of the classes of XPCOM components that can be constructed or accessed as an XPCOM service. Query Interface Let’s recall QueryInterface function. From reading this MDN article The function QueryInterface(...) is a function provided by all components which can be used to get a specific interface of that component. This function takes one parameter, the interface that you want to get. and from reading Creating Applications with Mozilla book, QueryInterface A method used to match an interface with a desired implementation. The implementation can be in C, C++, JavaScript, Python, and other languages for which appropriate bindings are created. You can have multiple implementations for an interface. Through QueryInterface, you can ask for and assign the desired interface to its implementation. Each XPCOM object needs to implement QueryInterface in order to return an instance of that object’s class. And also, tangentially related, I learned more about Components object. According to the above book, In XPConnect, XPCOM interfaces, classIDs, and progIDs are stored as global JavaScript objects and properties that can be manipulated directly through a top-level object called Components. This object accesses any component that is declared “scriptable” in an XPCOM IDL interface. Through the Components object, you can access and use the services that these interfaces provide. The Component object’s top-level properties and methods include: QueryInterface, interfaces and more! You might have guessed it but here is what exactly interfaces are: interfaces A read-only object array containing all the interfaces declared scriptable in the IDL file. The object name has the same name as the interface it represents. Confusion I have been reading a lot about XPCOM and I am forgetting the difference between components, classes, services. Some of the functionality that the browser provides needs to be used by both consumers in core browser written in C++ and by the front-end of the browser which is written in JavaScript. XPCOM provides us with a way to implement such functionality in one language and expose it to a different language. There are clearly defined interfaces, which are language agnostic and they are specified using IDL, and they attempt to outline different functionality that a browser can provide. A component is a reusable piece of code that implements such an interface (one or more of such interfaces). Some of the components need to be instantiated every time they are used and they are called classes, and others exist as singleton instances and don’t need to be invoked every time they are used and they are called services. Thoughts In Progress So now we see that in the above code example, Services.io needs to return a nsIIOService service that also has an interface nsISpeculativeConnect. Let’s check if we can find a class that implements both such things, to see if I understood correctly everything I just read. Indeed, nsIOService implements both. Now, I need to think about how I will make sure that the correct class will be returned when the getter is called. I’m going to look at the implementation of QueryInterface. Update My mentor pointed me towards this code so I will be doing something like this if (IS_WN_REFLECTOR(obj)) auto tearOff = XPCWrappedNative::Get(obj)->FindTearOff(*iid); to get a tear off object which will allow us to access both interfaces. More updates I kept trying to include xpcprivate.h but the compiler kept giving me an error ... Services.cpp:31:10: fatal error: 'XPCWrappedNative.cpp' file not found Then I realized I need to modify xpcom/build/moz.build so that it will include the appropriate files in the build.