0. Picking a topic area

I have “decided” to do my area exam many times before now. Area exams are easy to postpone, because as PhD students it’s easy to feel like there’s always something more urgent to be working on. Over the years I have repeatedly fallen in to the trap of thinking of myself as a code- and paper-writing machine (which means no papers === I’m a complete failure! This machine is broken! Now I need to write twice as fast to make up for my humiliating failure! Etc.).

But besides the institutional requirement to get it done, the main thing giving me forward momentum while working on my exam this time is having a clear idea of what I want to discuss. The area exam talk is about 40 minutes, which is both intimidatingly longer than any talk I’ve ever given and shorter than I expected. You’re not going to be able to sum up, say, all of neurosymbolic programming or planning for physical HRI in 40 minutes, and trying to do so will both waste your time and give you unnecessary anxiety. My topic is something that I have at least started some research work in, so I finally feel I have an okay grasp on what’s relevant and what’s not.

My topic is AI for the facilitation of task handover, which is (1) honestly still too broad for a 40-minute talk, but I did my best and refined as I went, and (2) an interesting choice because AI is not currently used to facilitate task handover (at least, not by anyone but me). The goal of the area exam is to demonstrate your understanding of the state of the art in your chosen area. Because my area is an application domain rather than a technique, and a currently-unrealized application domain at that, my talk will aim to look at the state of the various component technologies that would be applied to this problem space. This (in theory) lets me look at huge areas like LLM summarization and pick out the current papers that are most relevant to my desired application.

1. Deciding to do the area exam, and setting a timeline

If you have completed step 0, this should really only be the logistics step. Talk to your advisor to finalize your topic and, with their help, figure out 2 other faculty members that care about that topic and ask them to be on your committee. (This request does not have to be a big to-do – a fairly short email with a description of your topic and a brief explanation of why you think they would be good to have on your committee given their expertise in XYZ worked just fine for me).

Schedule yourself a solid ~3 weeks where you can focus mostly or completely on the exam, then send out a when2meet or equivalent to your commitee for some 1-2 week period after that. Once you’ve confirmed everyone’s availability, send out that calendar invite, let your labmates know, and book a conference room (at CU you can do this at CU EMS, or by contacting the CS front office directly to book a room in ECES or DLC). I’d strongly recommend also scheduling a practice talk a few days beforehand and inviting labmates and/or friends.

In terms of a progress timeline, your major milestones should probably be something like:

a. Finalizing a paper list with your advisor: 2 weeks before exam b. Sending your advisor your full slide deck for feedback: 1 week before exam c. Giving a practice talk to your lab to get more feedback: 2+ days before exam

If at all possible, schedule multiple meetings with your advisor ahead of time for discussion and feedback, especially for steps (a) and (b).

2. Making your outline

I had already done a fair amount of reading on all the component areas I wanted to touch on, but didn’t know exactly what papers I would be discussion or what structure I would use. After much consideration and scrapped attempts, I ended up taxonomizing the component parts of the task handover problem this way:

a. forming a state estimate from input data b. acquiring a listener model for producing model updates c. performing model updates given a difference in mental models

My advisor initially recommended I pick 2 relevant papers for each of these problem spaces, so 6 papers total. Note that this is a pretty uncommonly small number of papers; in most prelim exams I typically see more in the 8-12 range, sometimes more. Again, my situation is somewhat unique in that I’m using a potential application area as a lens to discuss relevant findings. That means that each paper needs to be dicussed in a fair bit of depth, because I need to spend more time connecting it back to the topic at hand.

Lastly, I came up with a few broad thesis statements that I would use to frame each of my papers:

1. Recent advances in related areas of AI can be applied to task handover​
2. LLMs have made these advances more applicable to real-world communication scenarios

These are hardly the most controversial or specific theses, but having them was helpful for me in motivating my narrative and I’d highly recommend it. You can fall back on your taxonomy and have something as simple as “each of these N approaches has its own strengths” or “X new technology has enabled us to take this field in Y direction” or “Z is an ongoing problem that remains unsolved in this area, and here’s what we know about it / how we have progressed towards solving it.”

3. Choosing your papers

At our next meeting my advisor told me to focus my 6-paper main lit review on the first two areas, and leave the last section – “performing model updates” – as an abbreviated sort of current-work teaser at the end. This gave me room to discuss 2 more papers in the “listener model acquisition” section.

For each paper, it’s really important that you have a clear idea of what that paper’s contributions are, and why your listeners should care about it. There are also other aspects to consider.

• While you’re definitely going to learn new things during this process and step outside of your comfort zone somewhat, you also don’t want to be choosing papers that are wildly outside your area of expertise. Remember the goal here is to show off the expertise you’ve developed in your PhD so far – leverage what you already know about as much as possible.
• Relatedly, you might have already done some research work in your topic area, so use this as an opportunity to highlight what prior work has been most relevant to you as a scientist in this area.
• Avoid including papers that are more than a few years old, unless you have good reason (e.g. they were particularly impactful, laid the foundation for future papers you’re also going to discuss, etc.). AI moves very fast, and even papers from 2-3 years ago can become quite outdated. Try looking at works that have cited a relevant paper instead to see what’s been done more recently.
• Avoid including papers that your committee members authored. This is tempting if they’ve done relevant work, but you’re setting yourself up for a potentially uncomfortable Q&A session.

For an example, the final paper list my advisor approved looked like this:

1. Obtaining a model of state and environment (i.e., task-relevant state) (n.b.: don’t focus on environment dynamics)
   • Zhang et al. Hierarchical Cross-Modality Semantic Correlation Learning Model for Multimodal Summarization - a recent-ish (2022) and well-cited paper that had a lot of success on summarization metrics by using semantic correlation learning to combine image and textual data into a single representation and text output. This paper showcases one of many strategies that can be used to extract meaning from multimodal input, making AI better able to recover task relevant information from the world around it. Key point: We can extract meaningful from multiple modalities (for developing a state estimation)
   • Das, Chernova & Kim, NeurIPS 2023. State2Explanation: Concept-based explanations to benefit agent learning and user understanding. Cited by 24. Learns an embedding model between agent states and associated concept-based explanations and leverages such learned model to (1) inform reward shaping for agent learning, and (2) improve end-user understanding of the agent’s decision making at deployment. Key point: We can create an explanation-grounded internal model that can then be used to generate future explanations
2. Obtaining a model of the receiver (i.e. the human model) – giving a model update requires knowing what model you’re updating common thread: what does this paper mean in the age of LLMs? What’s the lesson we can transfer into an LLM-powered method?
   • Zhang & Soh, Large Language models as zero-shot human models for HRI. https://ieeexplore.ieee.org/abstract/document/10341488 Key point: In some situations, we may not need a bespoke human model
   • Salomons, Akdere & Scassellati. IJCAI 2021. BKT-POMDP: Fast Action Selection for User Skill Modeling over Tasks with Multiple Skills.
   • Yang et al., Machine Learning and Knowledge Discovery in Databases 2020. GIKT: A Graph-based interaction model for knowledge tracing. https://link.springer.com/chapter/10.1007/978-3-030-67658-2_18 Cons: IDK this conference. Pros: 150+ citations so clearly it was impactful. Contrasts with Nicole’s paper above by using a graph convolutional network for knowledge tracing, showcasing a different technical approach that might be more appropriate in some scenarios
   • Chen, Fong & Soh, MIRROR: Differentiable Deep Social Projection for assistive HRC https://arxiv.org/pdf/2203.02877 Key point: Rather than knowledge tracing, this system learns observational & policy residuals in advance, then uses that to predict what human is and is not aware of and what their tendencies are
3. Delivering model updates: Conclusion, touching on my current work & a few related works (very briefly)
   • Chakraboti et al, 2017. Plan Explanations as model reconciliation: Moving beyond explanation as soliloquy. Key point: proposing computational approaches to various explanation types, given a human model
   • Tabrez, SPEAR (policy elicitation). SPEAR uses the goal of policy elicitation to determine how best to update the user’s reward function via Boolean algebra and an MIL to optimize for coverage and conciseness. Key point: Builds on the above and showcases a particular approach to model reconciliation via policy elicitation

This took a non-neglible amount of back-and-forth with my advisor to settle on, so talk with them early & often in this process for best results!

4. Making your slides

I am by no means a genius at PowerPoint creation, so your mileage may vary, but here’s what worked for me.

• Start with a sketch of your presentation’s structure. I made a new PowerPoint and literally just wrote titles for each slide to get a sense of what I was going to talk about and where, and put things like “[illustrative figure goes here]” for my future self to figure out.
  • Intro: where you give the necessary background on your research area and why people should care about it.
  • Thesis: Some kind of takeaway from all your papers and why you’re talking about them. What do you want your listeners to be convinced of by the end of your talk? If they remember 1-2 things, what should they be?
  • Roadmap: Introduce the taxonomy you came up with, if you have one, or at least some way of grouping your papers to give your talk structure.
  • Lit review: Go section by section from your outline, and then paper by paper. For each one, you won’t have time to give a full in-depth overview of what the authors did – instead you’ll have only a few slides (maybe only 1, depending on how many papers you have!) to say very specifically what’s unique about the paper and how it relates to your thesis. At the end of each section, I recommend having a quick recap slide to summarize the key points from the papers you just discussed.
  • Conclusion: This is where you can briefly talk about your current work, where you think the field is going & how you want to contribute to it. Where do you fit in in the story you’ve just told?

• Build out your intro (and maybe even conclusion) next_, because it will force you to get a very solid idea of the perspective you’re taking on each of your papers. (I actually did slides for my papers first, then went back after I wrote my intro and realized I wanted to rework literally all of them. Don’t be like me.) This might be where you have to make some figures or illustrations as well, to convey your problem space and outline visually. In your conclusion you might also get to talk about your own past or present work.

• Build out your paper slides: Honestly the easiest part, but some things to keep in mind:
  • Include citation information where appropriate, including for images.
  • Pictures are worth a thousand words – if the paper has decent figures, use them. But–
  • If you’re going to use a figure from the paper, it should either be completely self-explanatory (as in, very simple / just a photograph / etc.) OR you should take an appropriate amount of time to contextualize it or reference it directly in your talk.
  • Same goes for any math, data tables, etc.: if it’s on your slide, you should speak to it. (That’s a Brad Hayes-ism, for anyone curious.) Don’t use them just for set dressing!
  • It’s okay if your slides are very text-heavy on the first draft, but eventually you’ll want to minimize on-slide text – you want people listening to you, not reading. Optimize for clarity, spread info across multiple slides if needed.
  • Similarly, complicated figures like system diagrams or tables can be overwhelming and/or distracting for your audience, even if you explain them. Include only the most relevant figures/parts of figures, and consider doing ‘progressive reveals’ where you un-hide parts of the figure/text as you discuss it to help listeners follow along. This was suggested to me by a labmate at my practice talk, and now I’m a big progressive reveal fan and I think everyone else should be too.
  • The key takeaway from each paper should be written out, super explicitly, on the slide – listeners should be 100% clear on why you care about that paper and they should too.
• Check your clarity & time. Are you going to be able to get through all those slides in 40-45 minutes, without rushing through? As I wrote my presentation, I timed myself doing a quick-and-dirty run through of each section to get a sense of where I was at. (For example, my initial pass at my intro took me about 15 minutes(!), so obviously I had some trimming to do.) This is also where I condensed text where possible, and moved a lot of text off my slides and into my presentation notes.

Once you have a rough draft, it’s time to send the slides to your advisor for feedback & approval.

5. Prepping & practicing your talk

I relied heavily on the Presenter View feature in PowerPoint (also in Google Slides) so that I could reference my notes on my laptop while I was speaking. I tried to limit these to bullet pointed almost-sentences that still covered everything I wanted to say at each slide. Still, you don’t want to be reading off your laptop the whole talk. It took me a few practice runs to get the talk to a half-memorized state, where I could take quick glances at each bullet point to know what to say next.

(I still definitely read off my screen excessively because my memory fails me under pressure and I dislike eye contact. It’s important to know your weaknesses.)

The practice talk I gave for my labmates was incredibly helpful: for the practice, for the feedback they gave, and for the validation of something I’d been working on for weeks. Even though I was nervous to give it, the confidence boost it gave me going into my actual exam was extremely worth it. They also asked some great general questions that made me feel more prepared for the Q&A.

6. Giving your talk

At this point, your advisor has ok’d your papers and your slides, your labmates have ok’d your talk, you’ve practiced – your success is more or less guaranteed. Try not to be nervous! Speak clearly and confidently. Give you (and your audience) time to breathe; have some water nearby and take a sip between slides as needed.

For the closed Q&A session, my committee mercifully did not grill me on any technical details. Their questions were really testing my ability to articulate my opinions and decision-making: about what I included in my talk, what is or is not exciting about a particular paper, what the biggest challenges are, and so on. I came out of it feeling pretty good, which was a pleasant surprise.

7. The aftermath

I passed! You can view the slides from my talk here. If you’re a CU affiliate you can view a recording of it here.

I’m so grateful to have gotten through this process with my pride mostly intact. I’m also very thankful to my committee for their patience with me – it took me a long time and a lot of iteration to get the presentation to a state we all were happy with. I wrote this post in hopes that it would help make the process a bit less intimidating for future students. Good luck!

It’s been a while! I had a lot of grand ambitions for keeping up this blog consistently, but obviously that did not come to pass. The fact is that a lot has happened since mid-2022! The project I was working on at that time ended up being kind of a disappointment (I can blame the nature of human subjects experiments until I’m blue in the face… and I will). Since then I made a substantial pivot in my research direction, focusing less explicitly on education (though it remains a topic of great importance to me!) and more on how robots and AI can act as embedded cognitive supports in other environments. Accordingly, I transitioned away from my work with the inter-departmental institute I had been working with at the time, and suddenly found myself operating even more independently, with an overwhelming amount of freedom to pursue whatever I wanted.

I also was diagnosed with a chronic illness, took a leave of absence to get a major surgery, got COVID again, dealt with some family health problems, continued to struggle with my mental health and severe imposter syndrome, got diagnosed with ADHD and started medication, and generally did a lot of floundering when it came to not only my research direction but my life trajectory, passions, and values. Ah, the joys of young adulthood. I’m very grateful to my advisors for allowing me the space and time to take care of my physical & mental health, and helping me find a research thread that I’m passionate about.

Another big change that’s happened since 2022 is LLMs suddenly being everywhere and everything. The nature of HAI and HRI work has completely changed as a result. With LLMs and multi-modal language models (MLMs) suddenly on every app and website you can imagine, I think it’s become apparent that these models are extremely powerful – but also that they are not a magic bullet or a solve to general intelligence. This, to me, is more exciting than the alternative! We have this new powerful tool whose limitations we are still working to discover and mitigate. How do we wield these models to their fullest potential? As a spoiler, I think the answer lies in leaning into LLM’s strength – lingustic operations! – and combining them with other AI techniques to get both the usability of LLMs and the interpretability and consistency that we want in our AI (or at least I do). This is something we’ve already started to see from major LLMs like ChatGPT: by incorporating retrieval-augmented generation, GPT is now able to source its claims from a live knowledge base (i.e. the internet), rather than relying on implicit knowledge gleaned from outdated training data to answer user questions.

Right now, my research topic is in the form of a target application area – task handover (no, not literal object handover from a robot). The inspiration for this topic was actually patient handover in hospitals. This is a difficult cognitive task that nurses and other clinicians must perform at the end of every shift. The outgoing nurse must provide the incoming nurse with all of the relevant information that they need to ensure continuity of care for the patient, for each patient. This info can include aspects of the patient’s medical history, active conditions, the outgoing nurse’s action history, any tests the patient might be waiting on, conditional recommendations… the list goes on. Despite the proliferation of electronic health records and standardized handover protocols, existing systems still fail to offload the cognitive and temporal burden faced by clincians in maintaining and delivering this information.

So, that’s what my area exam (and probably, eventually, my dissertation!) is going to be about: how can AI support the handover process? What advances in things like language models, sensing, and model reconciliation can be applied to this challenging problem space?

Now I’m going to start writing about the process of doing this area exam. Wish me luck.

There are lots of great C++ development environments out there. But for me, because I’m studying how robots can guide users through the debugging process, I specifically needed an interactive C++ editor and testbed that can communicate with my robot’s backend. This way the robot can be aware of what edits the user is making, what error messages they get, when they successfully compile (or not!), and so on. So, below I’ve detailed my process of creating this little app.

0. Software prerequisites

• gdb – a debugger program for C/C++
• g++ – my C++ compiler of choice
• tkinter – a python library for GUI apps
• pygdbmi – a python library for controlling gdb

1. Text Editor

The first step is to create the text editor window, where users can open files, edit code, and save them. I based the bulk of my starter code off of this tutorial, but things got a bit more complicated when I wanted to add line numbers to the text window. I used the TextLineNumbers and CustomText classes from this newbedev post:

	from tkinter import *
	class TextLineNumbers(Canvas):
		# module to add line numbers
		def __init__(self, *args, **kwargs):
			Canvas.__init__(self, *args, **kwargs)
			self.textwidget = None
		def attach(self, text_widget):
			self.textwidget = text_widget
		def redraw(self, *args):
			'''redraw line numbers'''
			self.delete("all")
			i = self.textwidget.index("@0,0")
			while True :
				dline= self.textwidget.dlineinfo(i)
				if dline is None: break
				y = dline[1]
				linenum = str(i).split(".")[0]
				self.create_text(2,y,anchor="nw", text=linenum, font=("Courier New", 12))
				i = self.textwidget.index("%s+1line" % i)
	class CustomText(Text):
		# text widget with event proxy
		def __init__(self, *args, **kwargs):
			Text.__init__(self, *args, **kwargs)
			# create a proxy for the underlying widget
			self._orig = self._w + "_orig"
			self.tk.call("rename", self._w, self._orig)
			self.tk.createcommand(self._w, self._proxy)
		def _proxy(self, *args):
			# let the actual widget perform the requested action
			cmd = (self._orig,) + args
			result = self.tk.call(cmd)
			# generate an event if something was added or deleted,
			# or the cursor position changed
			if (args[0] in ("insert", "replace", "delete") or 
				args[0:3] == ("mark", "set", "insert") or
				args[0:2] == ("xview", "moveto") or
				args[0:2] == ("xview", "scroll") or
				args[0:2] == ("yview", "moveto") or
				args[0:2] == ("yview", "scroll")
			):
				self.event_generate("<<Change>>", when="tail")
			# return what the actual widget returned
			return result    

Next, I put it all together to make a class for the editor window:

	import os
	class EditorWindow:
        """ Module to draw the code editor window, with line numbers.
            Inputs:
                root: the tkinter root (a Tk() instance)
                edit_cb : a function be triggered when user makes changes.
         """
		def __init__(self, root, edit_cb):
			self.external_bindings = {}
			self.root = root
			self.root.title("TK Editor")
			self.root.geometry("1200x800")
			self.root.resizable(True, True)
			self.root.columnconfigure(0, weight=1)
			self.root.rowconfigure(0, weight=1)
			self.frame = Frame(self.root, bd=2, relief=SUNKEN)
			menu_bar = Menu(self.frame)
			self.file_name = None
			# Adding the File Menu and its components to create Python Text Editor
			file_menu = Menu(menu_bar, tearoff=False, activebackground='DodgerBlue')
			file_menu.add_command(label="New", command=self.open_new_file)
			file_menu.add_command(label="Open File", command=self.open_file)
			file_menu.add_command(label="Save As", command=self.save_file)
			file_menu.add_separator()
			file_menu.add_command(label="Close File", command=self.exit_application)
			menu_bar.add_cascade(label="File", menu=file_menu)
			# Adding the Edit Menu and its components
			edit_menu = Menu(menu_bar, tearoff=False, activebackground='DodgerBlue')
			edit_menu.add_command(label='Copy', command=self.copy_text)
			edit_menu.add_command(label='Cut', command=self.cut_text)
			edit_menu.add_command(label='Paste', command=self.paste_text)
			edit_menu.add_separator()
			edit_menu.add_command(label='Select All', command=self.select_all)
			edit_menu.add_command(label='Delete', command=self.delete_last_char)
			menu_bar.add_cascade(label="Edit", menu=edit_menu)
			self.root.config(menu=menu_bar)
			scroller = Scrollbar(self.frame, orient=VERTICAL)
			scroller.pack(side=RIGHT, fill=Y)
			self.text_area = CustomText(self.frame, font=("Courier New", 12), undo=True)
			self.linenumbers = TextLineNumbers(self.frame, width=60)
			self.linenumbers.attach(self.text_area)
			self.linenumbers.pack(side=LEFT, fill=Y)
			self.text_area.pack(side=RIGHT, fill=BOTH, expand=1)
			scroller.config(command=self.text_area.yview)
			self.text_area.config(yscrollcommand=scroller.set)
			self.text_area.bind("<KeyPress>", edit_cb)
			self.text_area.bind("<<Change>>", self._on_change)
			self.text_area.bind("<Configure>", self._on_change)
			self.frame.pack(side=TOP, fill=BOTH, expand=True)
		def _on_change(self, event=None):
			self.linenumbers.redraw()
		def get_contents(self):
			return self.text_area.get(1.0, END)
		def _open_file(self, filename):
			with open(filename, "r") as file:
				self.text_area.insert(1.0, file.read())
				file.close()
			self.file_name = filename
		# Creating all the functions of all the buttons in the NotePad
		def open_file(self):
			file = fd.askopenfilename(defaultextension='.cpp', filetypes=[('All Files', '*.*'), ("C++", "*.cpp"), ("Text File", "*.txt*")])
			if file:
				print(file)
				self.root.title(f"{os.path.basename(file)}")
				self.text_area.delete(1.0, END)
				self._open_file(file)
			else:
				file = None
		def open_new_file(self):
			self.root.title("Untitled - Editor")
			self.text_area.delete(1.0, END)
		def save_file_as(self):
			file = fd.asksaveasfilename(initialfile='Untitled', defaultextension='.cpp',
										filetypes=[('All Files', '*'), ('C++', '.cpp')])
			f = open(file, "w")
			f.write(self.text_area.get(1.0, END))
			f.close()
			self.file_name = file
			self.root.title(f"{os.path.basename(file)} - Editor")
			print("Saved to ", self.file_name)
		def save_file(self):
			if self.file_name is None:
				self.save_file_as()
			else:
				f = open(self.file_name, "w")
				f.write(self.text_area.get(1.0, END))
				f.close()
		def exit_application(self):
			self.root.destroy()
		def copy_text(self):
			self.text_area.event_generate("<<Copy>>")
		def cut_text(self):
			self.text_area.event_generate("<<Cut>>")
		def paste_text(self):
			self.text_area.event_generate("<<Paste>>")
		def select_all(self):
			self.text_area.event_generate("<<Control-Keypress-A>>")
		def delete_last_char(self):
			self.text_area.event_generate("<<KP_Delete>>")

If you initialize tkinter and run this in a mainloop(), you get the following result:

2. Output Window

In addition to the editor window, I wanted my program to launch a separate window where any messages from the robot’s backend, like error messages from the C++ code, could be displayed. I also wanted to add a Test button that would test the code (once I had implemented compiling & testing!). Lastly, I wanted a text entry box where the user could input text that would (again, once I had done the necessary connectivity) communicate to their program.

	class OutputWindow:
		""" Module to draw an output display window, plus text entry and a test button.
			Inputs:
				root : the tkinter root (a Tk() instance)
				test_fn  : a funciton to be triggered when user presses the "Test" button.
		"""
		def __init__(self, root, test_fn):
			root.title("Output")
			root.resizable(True, True)
			root.columnconfigure(0, weight=1)
			root.rowconfigure(0, weight=1)
			frame = Frame(root, bd=2, relief=SUNKEN)
			bottomframe = Frame(root, relief=SUNKEN)
			self.text = Text(frame, font=("Courier New", 12), state=DISABLED)
			self.text.pack(side=LEFT, expand=True, fill=BOTH)
			scrollbar = Scrollbar(frame)
			scrollbar.pack(side=RIGHT, fill=Y)
			scrollbar.config(command=self.text.yview)
			self.text.config(yscrollcommand=scrollbar.set)
			self.scrollbar=scrollbar
			test_button = Button(bottomframe, text="Test", command=test_fn)
			test_button.pack(side=RIGHT)
			self.user_input = StringVar()
			self.last_input = None
			self.entry_field = Entry(bottomframe, font=("Courier New", 12), textvariable = self.user_input)
			self.entry_field.bind("<Return>", self._get_input)
			self.entry_field.pack(side=LEFT, fill=X)
			frame.pack(side=TOP, fill=BOTH, expand=True)
			bottomframe.pack(side=BOTTOM, fill=X)
		def _get_input(self, event=None):
			user_input = self.user_input.get()
			self.entry_field.delete(0, END)
			self.place_text(user_input)
			if not user_input:
				self.last_input = None
			else:
				self.last_input = user_input
		def get_input(self):
			line = self.last_input
			self.last_input = None
			return line
		def place_text(self, new_text):
			# self.test_count += 1
			prev_yview = self.text.yview()
			self.text["state"] = NORMAL
			self.text.insert(END, new_text + "\n")
			self.text["state"] = DISABLED
			self.text.yview_moveto(prev_yview[1])

Note that I’m not doing anything with the input text from the Entry box yet – just storing it in an attribute, where it can be retrieved later via get_input. Once I got the above running and displayed some sample text, it looks like this:

3. Compiling & Testing

Next, I want the “Test” button to actually compile the C++ code so it can be run by the test code I’ll implement later. The compilation itself is as simple as calling g++ via subprocess:

	import subprocess
	def compile(cpp_file, binary_file):
		""" Compiles the cpp file to an executable of name binary_file.
			Inputs:
				cpp_file: C++ filename to be compiled.
				binary_file: desired name of compiled binary file produced.
			@return: "(True, "")" if compiled successfully
					 else, (False, <error>) where <error> is the stderr message returned by g++
		"""
		res = subprocess.run(["g++", "-g3", cpp_file, "-o", binary_file], check=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
		print(res.stdout.decode('utf-8'))
		if res.returncode != 0:
			return (False, res.stderr.decode('utf-8'))
		return (True, "")

Running the compiled program can be more difficult. I started by using a fairly simple call to pygdbmi’s GdbController.

	from pygdbmi import GdbController
	gdbmi = GdbController()
	response = gdbmi.write('-file-exec-and-symbols ' + binary_file)
	response = gdbmi.write('-exec-run')

While this gives me the output that I’ll need to update the output window display, I also want to be able to communicate with the program while it’s running via text entry. In addition, I want to parse the response object (a list of dicts) to separate GDB’s error messages from the program’s intended output.

To implement all this, I wrote a class that launches the Editor and Output windows, and implements compilation, testing, and interactivity with programs. The run_test function is passed into the Output window to be triggered when the user presses “Test”.

	class CppEditorNode:
		def __init__(self):            
			tk     = Tk()
			self.editor = EditorWindow(tk, self.edit_cb)
			self.output = OutputWindow(Toplevel(), self.run_test)
			self.test_count = 0
			self.editor._open_file("/home/kaleb/code/ros_ws/src/robo_copilot/assets/test1.cpp")
			self.in_debug = False
			self.gdbmi    = None
			while True:
				try:
					if self.in_debug:
						self.monitor_test(self.gdbmi)
					self.tk.update()
				except:
					break
			tk.quit()
		def compile(self, cpp_file, binary_file):
			res = subprocess.run(["g++", "-g3", cpp_file, "-o", binary_file], check=False, stdout=subprocess.	PIPE, stderr=subprocess.PIPE)
			if res.returncode != 0:
				return (False, res.stderr.decode('utf-8'))
			return (True, "")
		def run_test(self):
			if self.in_debug:
				return
			self.editor.save_file()
			self.test_count += 1
			
			cpp_file = self.editor.file_name
			binary_file = os.path.join(
				os.path.dirname(self.editor.file_name), "tmp.out"
			)
			
			result, err = self.compile(cpp_file, binary_file)
			if not result:
				# failed to compile
				msg.type = msg.COMPILE
				msg.stderr = err
				msg.stdout = ""
				self.test_pub.publish(msg)
				self.output.place_text(err)
				return
			# compiled successfully, now run via gdb
			self.gdbmi = gdbmi = GdbController()
			response = gdbmi.write('-file-exec-and-symbols ' + binary_file)
			response = gdbmi.write('-exec-run')\
			
			if self.process_gdb_response(response):
				self.in_debug = True
			else:
				self.in_debug = False
				self.gdbmi = None
				gdbmi.exit()
		def process_gdb_response(self, response):
			""" Parse and appropriately output gdb data from param response.
				@return: True if gdb process is still running, else False
			"""
			msg = Error()
			program_output = ""
			gdb_output = ""
			for item in response:
				if item["type"] == "output":
					program_output += item["payload"]
					self.output.place_text(item["payload"])
				elif item["type"] == "console":
					gdb_output += item["payload"]
					self.output.place_text("\nDEBUG: " + item["payload"])
				elif item["message"] == "stopped":
					if item["payload"]["reason"] == "exited-normally":
						msg.type = msg.SUCCESS
					else:
						msg.type = msg.RUNTIME
						msg.payload = item["payload"]
					return False
			# program is presumably waiting for user input
			return True
			
		def monitor_test(self, gdbmi):
			# check for input
			user_input = self.output.get_input()
			if user_input is None:
				return
			response = gdbmi.write(user_input, raise_error_on_timeout=False)
			# parse new response, and see if program is still running
			self.in_debug = self.process_gdb_response(response)