Applying HAXM to Google Summer of Code

[VelocityRa]

Have you guys considered applying for Google Summer of Code?

You'd need a HAXM maintainer to devote a bit of time on being a mentor (can be a few minutes or so per day, just to check on progress and/or answer questions) and in return you get 3-months of someone (or more people) working on HAXM, full-time.

Personally I'd be interested if you did apply, and I can't be the only one.

[krytarowski]

@VelocityRa are you interesting as a student?

[VelocityRa]

@krytarowski Yes, I'd be interested to work on haxm as a student. It'd be my second GSoC as such.

[krytarowski]

@VelocityRa I'm a mentor under The NetBSD Foundation umbrella and if you are interesting in enhancing the NetBSD as host support I can mentor you.

We can already boot and to some extent use FreeDOS (example: https://twitter.com/sehnsuc77565442/status/1072182648125251584), Linux/BSDs still cannot be booted and I suspect that there are just 1-3 silly bugs somewhere. Future work is about enhancing the support like probably proper handling of debug registers, getting qemu debugger server to work.

But first of all we need to get announcement of the mentoring foundations from Google.

[raphaelning]

@VelocityRa This is a great idea, thanks! For the GSoC application, it seems we'll need to come up with a list of project ideas (at least 4) for students. We'll work on that and submit an application in the next few weeks.

@krytarowski has already proposed a good project idea. If anyone has anything else (and wants to be mentor), please feel free to pitch in.

But first of all we need to get announcement of the mentoring foundations from Google.

What do you mean by mentoring foundations? I've been reading the GSoC Mentor Guide and it's very helpful.

[krytarowski]

What do you mean by mentoring foundations?

Google will pick (with unknown algorithm) a selection of open source organizations. New organizations probably can get up to 1 slot with a student.

Last year I've mentored 2 successful students and worked with many more candidates.

[raphaelning]

@krytarowski Cool, thanks for the information! Yes, after we submit our application, we'll wait for Google to announce this year's mentor organizations by the end of February.

[PatrickvL]

AMD support would be a valuable project

[raphaelning]

@PatrickvL Sure, if someone can volunteer to be the mentor (none of us at Intel can).

[VelocityRa]

Glad to see this was well received.
I actually had AMD (SVM) support in mind, but thanks for the suggestion @krytarowski, I will consider it, as well as the officially proposed ones.

[AlexAltea]

Really looking forward to this! Great idea, @VelocityRa!

AMD support would be a valuable project
Sure, if someone can volunteer to be the mentor (none of us at Intel can).

@PatrickvL @raphaelning For this, or other mutually beneficial projects that cannot be directly mentored by an Intel team, I'd be happy to be a GSoC mentor for HAXM. I'm relatively familiar with the codebase by now and could provide support to GSoC students.

The coding period "May 27, 2019 - August 19, 2019" works perfectly for me, and I could commit 1 hour per day for this matter. Due to conferences, 1 week at early-June (RECON) and 1 week at mid-August (DEFCON), I'd be less available than usual, but emails would still work, so I think we could make this happen!

@raphaelning Let me know if you want me to redact a proposal for AMD SVM support at all. Generally speaking, I think the biggest "selling point" for HAXM at GSoC it has become a truly cross-platform type-2 hypervisor (as mentioned in the Linux support patch) and how it could become a great platform for developers and researchers (as mentioned in the IOCTL patch... which I still haven't forgotten about!). 🙂

[raphaelning]

@AlexAltea Thanks a lot! I believe you'd be a great mentor :-) We'll publish a draft of the Ideas Page for GSoC for comments some time next week.

[raphaelning]

I've drafted two project ideas, and plan to add another 1-2 early next week. @krytarowski @AlexAltea Do you want to write up the ideas for NetBSD and AMD support? I guess I can write an outline, but I'm not sure about the scope of these projects. BTW, I was mainly referring to the official guidelines and QEMU's ideas page.

We need to submit our GSoC application by the end of next week (February 3). Google prefers a dedicated page for the ideas list, so once we've finalized the content, we'll create a new page in our GitHub wiki.

---

Develop a more useful haxm-check tool

haxm-check is a CLI tool included in every HAXM release for Windows (binary name haxm_check.exe) and macOS (binary name haxm-check). Its purpose is to check whether the host computer meets the system requirements for running HAXM (see below), and can be either integrated with the HAXM installer or run manually as a diagnostic tool. However, the current implementation only covers a few basic CPUID checks, which are not enough for today’s computers. For instance, on Windows, the tool may misleadingly report that the host CPU lacks support for Intel VT-x, when the real problem is that the user has not turned Hyper-V off.

The current tool is not open source, but since the code is trivial, it would be easier to start afresh. Your goal is to develop a new haxm-check tool that performs the following checks on the host:

• Hardware
  • CPU vendor and model
  • Whether Intel VT-x is supported by CPU
  • Whether Intel VT-x is enabled in BIOS
  • Whether EPT is supported by CPU
• Software
  • OS version and architecture
  • Whether Hyper-V is enabled (Windows only)

The new tool should be able to run on all Windows and macOS versions that HAXM supports. With a good design, it should be easily portable to other host OSes (e.g. Linux) as well.

• Skills required: Proficiency in C, C++ or Golang
• Possible mentors: @raphaelning, @wcwang
• Difficulty level: Easy

References:

1. System requirements for running HAXM
2. emulator-check tool (part of Android Emulator)
3. System Information tool for Windows (msinfo32.exe)

---

Develop a new HAXM installer for Windows

The HAXM installer is a friendly GUI application that helps the user install HAXM on their computer. It was excluded from the HAXM open source release for various reasons, one of which is that it uses some proprietary components. As an open source project, HAXM should have an open source installer that everyone can help improve.

Your goal is to develop an open source replacement for HAXM’s Windows installer. Conceptually, it is quite simple: because HAXM is a non-PnP (software only) driver, its installation does not require an INF file, and only involves the creation of a kernel driver service, as has been demonstrated by the HaxmLoader tool (see below). However, a production quality installer needs to be robust and user friendly. It should:

• Run pre-installation checks (existing installation, active HAXM guests, system requirements, etc.).
• Install different driver binaries for different Windows versions.
• In case of failure, tell the user what went wrong.
• Support uninstallation.
• Support silent (unattended) (un)installation.

Use of a Windows installer builder/framework might speed up development, but these tools usually come with a scripting language that you need to learn.

• Skills required: Proficiency in C, C++ or Golang
• Skills preferred: Experience with Windows installer frameworks (NSIS, WiX, msitools, etc.)
• Possible mentors: @raphaelning, @wcwang
• Difficulty level: Medium

References:

1. Current HAXM installer for Windows
2. HaxmLoader
3. KMDF non-PnP driver sample
4. NSIS
5. WiX

---

Enable persistent HAXM driver logs on Windows

On Windows hosts, the HAXM driver uses DbgPrint() to log debug and error messages. These messages are not written to any file, and therefore are permanently lost unless the user takes explicit action to capture them (e.g. by launching DebugView or WinDbg in advance). This makes it inconvenient for an ordinary user to provide diagnostic information, especially in case of a random HAXM driver bug. At least, the critical error messages (including the driver panic dump) need to go into persistent storage for retrieval at a later time.

Your goal is to enhance or even revamp the logging interface used throughout HAXM, and enable persistence for critical errors (or even better, for any type of message) on Windows. One way to do that is to create a system event log entry for each HAXM log entry, which will appear in Event Viewer.

• Skills required: Proficiency in C or C++, experience with Windows API
• Skills preferred: Experience with Windows driver development
• Possible mentors: @raphaelning
• Difficulty level: Medium

References:

1. Current HAXM logging helpers (Windows)
2. Capturing HAXM driver logs
3. DbgPrint
4. Logging driver errors
5. Event Tracing for Windows (ETW)

[krytarowski]

I've decided to spare time on it myself and submit it as a paper for a conference (deadline for registration is Jan 30th). I need HAXM sooner than later so I will try to finish it myself.

[raphaelning]

@krytarowski Sure, that's cool, thanks for the heads up!

…

On Fri, Jan 25, 2019, 10:57 PM Kamil Rytarowski ***@***.*** wrote: I've decided to spare time on it myself and submit it as a paper for a conference (deadline for registration is Jan 30th). I need HAXM sooner than later so I will try to finish it myself. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#153 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AElsbeRqzei5qdI4T5Qy_i_kGDz4aVp6ks5vGxtdgaJpZM4Z2eoH> .

[krytarowski]

@krytarowski Sure, that's cool, thanks for the heads up!
…
On Fri, Jan 25, 2019, 10:57 PM Kamil Rytarowski @.*** wrote: I've decided to spare time on it myself and submit it as a paper for a conference (deadline for registration is Jan 30th). I need HAXM sooner than later so I will try to finish it myself. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#153 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AElsbeRqzei5qdI4T5Qy_i_kGDz4aVp6ks5vGxtdgaJpZM4Z2eoH .

I've pushed the NetBSD support forwards and submitted patches as pull requests. It looks like the distance behind other OSes in support is shortened.

I've summarized the state on the NetBSD Foundation blog:
http://blog.netbsd.org/tnf/entry/the_hardware_assisted_virtualization_challenge