Adaptive Systems Research
          Group



Intelligent agents

(TME285)

LP III (third quarter), 2017


 

Basic information:

 

Lecturer and examiner:

Mattias Wahde, tel. 772 3727, e-mail: mattias.wahde@chalmers.se

 

Course assistants:

Sina Torabi, e-mail: sina.torabi@chalmers.se

Literature:

M. Wahde: Interactive partner agents - a practical introduction, CTH 2017

Preliminary program:


Lectures will take place on Tuesdays 8-10 and Wednesdays 13-17. The detailed schedule will be posted in early January.


Preliminary program:

The course is centered around projects that the students solve individually using C#. NET, as well as one group project. The lectures will cover the various topics in the compendium, but there will also be work sessions, in which students are expected to work on their assignments (using laptops) in interaction with the teachers.

 

Date

Time


Room

Contents

20170117

08.00-09.45


HA2

Course introduction and motivation. Brief descriptions of the topics covered in the course. Introduction to interactive partner agents (IPAs). Brief introduction to (distributed) programming in C# .NET (Chapters 1-2)

20170118

13.15-17.00


SB-H3

Background on C# .NET programming (Appendix A). Distributed programming (ii) (Chapter 2). Brief description of the C# .NET libraries used in the course (Chapters 3 - 8). Handout of assignments

20170124

08.00-09.45


HA2

Decision-making, memory, and dialogue in IPAs (Chapter 3)

20170125

13.15-17.00


SB-H3

Image processing for IPAs (Chapter 4) (2 hours). Work session (2 hours)

20170131

08.00-09.45


HA2

Advanced topics in C# .NET programming (2 hours)

20170201

13.15-17.00


SB-H3

Three-dimensional visualization and animation for IPAs (2 hours) (Chapter 5). Work session (2 hours). Group formation and specification of the group assignment [Mandatory attendance!]

20170207

08.00-09.45


HA2

Speech synthesis (Chapter 6) (2 hours)

20170208

--


--

No lecture

20170214

08.00-09.45


HA2

Speech recognition (Chapter 7) (2 hours).

20170215

13.15-17.00


SB-H3

Internet data acquisition (Chapter 8) (2 hours). Work session (2 hours)

20170221

08.00-09.45


HA2

Work session (2 hours)

20170222

13.15-17.00


SB-H3

Applications and examples of IPAs (1 hour). Work session (3 hours)

20170228

08.00-09.45


HA2

Work session (2 hours)

20170301

--


--

No lecture (Sina and Mattias available in their offices)

20170307

08.00-09.45


HA2

Group assignment demonstrations, second session (2 hours). [Mandatory attendance!]

20170308

13.15-17.00


SB-H3

Spare session for group assignment demonstration [Mandatory attendance, if this session takes place!]


Documents: 
20170117    Course memo
20170118    C# coding standard
20170118    Required structure for submitted assignments
20170118    Checklist for submission of assignments  
20170125    Minor code update (NOTE! The IPASrc.zip has not been updated. You must make this (small) update manually!)
20170125    An explanation of how to set up the C# solution for assignment DI1 (also useful for assignment DI2)
20170126    Solving a (potential) problem with the ObjectSerializer library Note: This is ONLY needed in case your C# IDE does not allow you to add references to the ObjectSerializer library (or complains that it cannot find that library). If you do not have that problem, you can just skip this document).
20170130    How to set up a C# application for solving an assignment


Slides from lectures:
20170117    Slides from Lecture 1

20170118    Slides from Lecture 2
20170124    Slides from Lecture 3
20170125    Slides from Lecture 4
20170131    Slides from Lecture 5
20170201    Slides from Lecture 6
20170207    Slides from Lecture 7
20170214    Slides from Lecture 8
20170215    Slides from Lecture 9
20170215    Sina's presentation on face detection
20170221    Information about the group presentations
20170222    Slides from Lecture 11
20170307    Information about a summer school in Japan

Assignments:
20170118    Assignments


Source code:
20170117    IPASrc.zip  NOTE! Set the number format on your computer so that it uses decimal "."! See Lecture 1, Slide 58.




FAQ:

Q1. How does one install a Windows partition on a MacBook?
A1. See https://support.apple.com/en-us/HT201468
  Note that the actual Windows is available for Chalmers students at the Student portal, see https://student.portal.chalmers.se/en/contactservice/ITServices/software/Sidor/software.aspx

Q2. How complex must the dialogue be for assignment DI1?
A2. The idea is that you should thoroughly familiarize yourselves  with (and use!) the agent structure define in the IPA libraries, and also to understand (and deal with) the various problems that occur when one implements a non-trivial dialogue. The dialogue need not be very complex, but it should at least be complete (unlike the very simple examples in TestAgent1 - TestAgent4, which bascially only illustrate the use of the various classes in the AgentLibrary), meaning that it should consist of a meaningful exchange of information (on the given topic) between the user and the agent. This will require handling various paths (in the dialogue) depending on the user's response, handling incomprehensible responses etc.

Q3. Do we need to implement additional classes for assignment DI1?
A3. It is not a requirement, but it is likely that you may wish to implement one or a few additional dialogue actions (derived from the base class DialogueAction; see also the slides from Lecture 3). The OutputAction and ResponseAction (available in the IPA libraries) will certainly be useful, but the other two available dialogue action classes (FindAction and ReadByTagAction) are simply meant to illustrate how one can (or rather will need to) implement additional dialogue actions for more complex dialogues.

Q4. It would be convenient to have a DialogueAction class that can match a partial input (substring) rather than having to provide an exact match, as is the case now, since the CheckMatch() method is placed in the base class DialogueAction.
A4. I agree. As mentioned before, I want to minimize the number of changes made in the IPA libraries during the course, in order to avoid confusion. However, this is a case where I believe that a change is warranted, so that you will be able to write your own dialogue action classes (derived from DialogueAction) in which you can match partial strings.
The required change (an simple addition of a single keyword in the DialogueAction class) is described in the Minor code update (20170125) under Documents above.

Q5. I'm experiencing problems with the ObjectSerializer library: C# does not allow me to add a reference to it.
A5. The likely cause of this problem is that your computer does not contain older versions of .NET (for which the library was generated). However, it is easy to fix. See this document.


Q6. I want to start with the image processing assignment (IP1, for example). Should I start from any of the demonstration applications that you have provided?
A6. No, in general for the assignments, you should write your own application (with a MainForm that you design etc.) from scratch (but you can of course have a look at the demonstration application to see how to add and use menus, buttons, text boxes etc. The dialogue assignments are a special case since, there, you can start from the corresponding demonstration application(s), as explained in the e-mail I sent on 20170125.  A good first step is to read Appendix A.1 and A.2 in the compendium, check the slides from Lecture 2, and also check various online sources, e.g. the links in the footnotes on p. 123 in the compendium. Moreover, before submitting any code, make sure to read all the relevant documents above, and to clean up the code (removing unnecessary test methods, debug output etc.)


Q7. In one of my applications, I want to save (and then later load) a C# object to a file. How do I do that?
A7. You should use the ObjectSerializer library, and follow the procedure described in Appendix A.7. Do not write explicit Load() and Save() methods (takes a long time, and easy to make mistakes) - instead, do use the ObjectSerializer library, which allows you to load or save an object with only a few lines of code. A usage example can be found in the FaceEditorApplication in the FaceVisualizationSolution (the event handlers for the Load and Save menu items in the File menu).


Q8. When I change the camera settings in the program distributed with the IPA libraries (the VideoProcessingApplication), the changed settings remain even when I run the camera outside the IPA source code. What should I do?
A8. I have not had this problem with any of my cameras or computers, so I'm not exactly sure why the camera keeps the modified settings. However, I would recommend that you start the laptop (it seems mostly that this problem occurs with built-in cameras), set appropriate camera settings from within the
VideoProcessingApplication, and then exit the program. After that, please use an external camera instead - you can get a simple webcam for around 100 SEK or less. Note! In order to run the correct camera (i.e. the external one), use the GetDeviceNames() method in the CaptureDevice class to obtain the correct device (camera), and then assign the DeciveName (in the camera) accordingly. Note also that it is normal that when you start a webcam, the settings are usually really bad, so you'll need to adjust them in the CameraSetupControl.


Q9. I can add a Viewer3D to my project (for Assignment VS1), but then I cannot compile it!
A9. You need to add a reference to OpenTK.GLControl (and OpenTK, if you have not already done it) in your project. Do the following: In Windows Explorer (not the C# IDE), browse to the /bin/Debug folder of the ThreeDimensionalVisualizationLibrary. Then, copy the OpenTK.dll and OpenTK.GLControl.dll files to the /bin/Debug folder of your program (i.e. the program in your AssignmentVS1 folder). Next open the C#
IDE, right-click on Add Reference (in your VS1 application), browse to the /bin/Debug folder (of the VS1 application), and add references to the two DLLs just mentioned. Then recompile (Rebuild solution) and run again.


Q10. How should we specify the file path when loading a file in our code (for example, a face XML for 3D visualization)?
A10. Well, ideally, you should not hard-code search paths, but instead specify (in your report) where to find the file(s) needed. However, if you do need to hard-code a search path (can be convenient during debugging, for example), make sure that it is a relative path, so that you can move the code (along with any data folders) to another computer without having to change the search path. You should add System.IO in the using clauses (of the .cs-file in question), and then you can get the path of the executable file as follows:

string applicationDirectory = Path.GetDirectoryName(Application.ExecutablePath);

You can then navigate to any other folder relative to this folder. For example,
to go three steps up in the folder structure, and then into a Data/ folder to open a file called File.xml, you can write

string dataDirectory = applicationDirectory + "\\..\\..\\..\\Data\\File.xml";

etc.

Q11. How can I center the 3D face (vertically) in the Viewer3D?
A11. Per default, the face goes from 0 to 1 in the vertical direction, whereas the viewer shows a range centered on 0. In order to center the face, just move it 0.5 units in the negative z-direction, i.e. face.Move(0,0,-0.5) (which will work provided, of course, that the various parts of the face, e.g. the eyes, are contained in the Object3DList of the face, so that the parts move with the face).


Q12. When I apply several consecutive operations in an instance of ImageProcessor, after doing edge detection (SobelEdgeDetect), nothing happens with the bitmap. How can this be solved?
A12. The problem is that the (original) image processor needs to relock its bitmap at the end of the SobelEdgeDetect() method. The final instruction (AssignBitmap(processedBitmap);) in that method does not lock the bitmap. Therefore, you must add a call to the Lock method at the very end of the SobelEdgeDetect method, so that it looks as follows:


...
   AssignBitmap(processedBitmap);
   Lock();  // <- Added line
}

NOTE: The same applies to the Convolve method (in ImageProcessor), so please add a final Lock() call there as well.


Last update: 20170307, 07.15