DD3501 Current Research in Proof Complexity Winter 2011/12

Results from Course Evaluation

    General questions about the course

  1. How did you follow the course?

    1. 88% (7 st) I was taking the course for credit and have done scribing and all problem sets (even if not all have been graded yet).
    2. 13% (1 st) I started taking the course for credit but did not finish it.
    3. 0% (0 st) I was a listener on the course.

  2. Did you find the course easy or hard?

    1. 0% (0 st) Very easy.
    2. 0% (0 st) Fairly easy.
    3. 25% (2 st) Medium.
    4. 25% (2 st) Fairly hard.
    5. 50% (4 st) Very hard.

  3. Did you find the course interesting?

    1. 50% (4 st) Yes, very interesting.
    2. 50% (4 st) Yes, fairly interesting.
    3. 0% (0 st) Neutral.
    4. 0% (0 st) No, not very interesting.
    5. 0% (0 st) No, not at all interesting.

  4. Did you understand sufficiently well at the beginning of the course what the course goals were?

    1. 75% (6 st) Yes.
    2. 25% (2 st) Don't know.
    3. 0% (0 st) No.

    If you think this would have been needed, how should the course goals have been explained better?

    Since the course goals were somewhat "open" at the beginning of the course, I think no one knows if they understood them sufficiently.

  5. In the prerequisites for the course, it said that "the main target audience are grad students and advanced undergrads. Some background in computational complexity theory and/or discrete mathematics will probably be helpful, but all that should really be needed is 'mathematical maturity' and a willingness to learn new stuff." Do you think this was a fair description?

    1. 88% (7 st) Yes.
    2. 13% (1 st) Don't know.
    3. 0% (0 st) No.

    What if anything would you have added to the course prerequisites?

    Though it might be obvious for a TCS course, computer programming skills may be added to the list.
    There were certain problems on the problem sets that required some knowledge of combinatorics that might be out of scope of the "common knowledge" of an undergrad. Apart from that I think the description is fair. Though I would maybe loose the word "probably", because background in those areas is definitely helpful :)

  6. Do you think you personally had the required background to be able to follow the course?

    1. 100% (8 st) Yes.
    2. 0% (0 st) Don't know.
    3. 0% (0 st) No.

  7. How did you hear about this course? (Check all alternatives that apply.)

    1. 63% (5 st) Word of mouth.
    2. 25% (2 st) Guest lecture by Jakob Nordström on other course.
    3. 0% (0 st) Lecturer on other course.
    4. 50% (4 st) Message on mailing list.
    5. 0% (0 st) Ad on bulletin board.
    6. 0% (0 st) Ad on the web.

    Information and interaction

  8. How often did you refer to the course webpages to find information about the course?

    1. 63% (5 st) Several times per week.
    2. 25% (2 st) Once or twice per week.
    3. 0% (0 st) Just a few times per month.
    4. 0% (0 st) Just a few times during the duration of the course.
    5. 0% (0 st) Essentially did not refer to the webpages at all.
    6. 13% (1 st) Not applicable.

  9. What do you think about the quality of the course webpages? (Were they kept up to date? Did they contain the information you needed? Was the information easy to find?)

    1. 75% (6 st) Very good.
    2. 25% (2 st) Fairly good.
    3. 0% (0 st) Acceptable..
    4. 0% (0 st) Fairly bad.
    5. 0% (0 st) Very bad.
    6. 0% (0 st) Not applicable.

    Any comments on the course webpages?

    Contentwise good. As for the structure, I would have preferred to find notes and problem sets more on the top, maybe integrated with the schedule table.
    It may be a good thing to split it up into several pages (one for the assignments, one for the biblio, etc) but it was very usable as it was.
    Excellent! The only thing to improve that comes to mind would be to make the schedule and scribe notes to a single list.
    The web pages were kept up to date to the same extent as the information on them, I think. Sometimes, they promised things that did not appear, but this was because these things were not ready yet.
    My use of the webpages was characterized by "bursts". While working with the problem sets I used the webpage several times a week, but in between I hardly used it at all.

  10. During the autumn term we had "opinion polls" several times with a few quick questions about the course that you were asked to answer. What do you think about these opinion polls?

    1. 13% (1 st) They were really good and/or seemed very useful.
    2. 63% (5 st) They were fairly good and/or seemed somewhat useful.
    3. 13% (1 st) Neutral as to the value of these polls.
    4. 0% (0 st) They were fairly bad and/or did not seem very useful.
    5. 0% (0 st) They were really bad and/or seemed useless.
    6. 13% (1 st) Not applicable.

    Any comments on the opinion polls?

    would be good to have more of them also in the second half
    Back in autumn I felt that it was a bit too early to say much and that the polls are too many. Often I did not have a strong opinion on the questions there. However, later during the course it was more the other way around.
    It keeps Jakobs promise of a "configurable" course
    They were probably good, but I don't remember them well enough to answer the question. This may be positive; it would be bad if the most memorable thing of the course was the opinion polls. :)


  11. How many of the lectures did you attend (22 all in all including guest lectures)?

    1. 0% (0 st) Less than 20%.
    2. 0% (0 st) 20-40%.
    3. 13% (1 st) 40-60%.
    4. 13% (1 st) 60-80%.
    5. 75% (6 st) More than 80%.

  12. What do you think about the number of lectures on the course?

    1. 0% (0 st) Way too many.
    2. 13% (1 st) A bit too many.
    3. 75% (6 st) About right.
    4. 13% (1 st) A bit too few.
    5. 0% (0 st) Way too few.
    6. 0% (0 st) Don't know — didn't attend lectures.

  13. What do you think about the regular (non-guest) lectures by Jakob Nordström from a pedagogical point of view? (Was the material explained well? Did the lecturer speak and write clearly?)

    1. 50% (4 st) Very good.
    2. 38% (3 st) Good.
    3. 13% (1 st) Acceptable.
    4. 0% (0 st) Fairly bad.
    5. 0% (0 st) Very bad.
    6. 0% (0 st) Don't know — didn't attend lectures.

    Any comments about the pedagogical contents of the regular lectures? (Bonus points for constructive criticism.)

    the recaps at beginnings of lectures were helpful. for lectures on the newer results, a bit more background / intuition would be nice. had been some back and forth on topics.
    From an objective perspective, I feel that the lecturer was well prepared and really tried to explain the material as well as possible. If I had the same presentation written on paper, or actually I have due to the (scribed) notes, it would be/is not too hard to follow the material. However, right back then in class, I personally often had problems in following all the time. This cannot really be blamed on the lecturer, it is just that with such a high level of the content, I need my time to rethink the single steps. That is very much depending on the individual. Unfortunately I cannot be very constructive here, as I don't know how to solve this problem. Clearly picking easier content would help, but that is not a real option, I suppose. Or maybe one could have more smaller lectures? Like one half in the morning, the other in the afternoon. On the other hand, having good notes provided, does permit the student to not understand everything in class.
    I can't see how you could cram that much information in a single lecture in any other way. If there would have been time over, a summary of the lecture at the end would have been nice. But there were very rarely any time over, so I think this was as optimal as it gets given the time constraints.
    It was very good, but usually rushed towards the end.
    They were planned so that we would know what to expect from them. The course notes were there so that we could see what the lecturer believed he was talking about. There were many occurrences of "this is what we are going to do" and "this is what we have done", but more of these would just be good.
    A special note on terminology: for the topics of this course, certain words like "efficient" seemed to have different meanings than what you might expect having taken some basic courses on computational complexity. It would have been nice to have this explicitly pointed out sometimes: look, now "efficient" means polynomial, now it doesnīt, this is the very specific meaning it has here. This led to some confusion as to whether "at most the same" meant asymptotically so or in exact numbers, or maybe just exponentially off in a specific way that we could not avoid anyway...

  14. In particular, what do you think about the pace of the regular lectures?

    1. 13% (1 st) Way too fast.
    2. 25% (2 st) A bit too fast.
    3. 63% (5 st) About right.
    4. 0% (0 st) A bit too slow.
    5. 0% (0 st) Way too slow.
    6. 0% (0 st) Don't know — didn't attend lectures.

    Any comments about the pace of the lectures?

    a bit too fast in the second half.
    This _relatively_ high pace is probably unavoidable with that high level content. I mean, with easier content, the pace would have been excellent, but as this material simply requires time for rethinking, internalizing, one would need 5 minutes breaks after each and every statement/proof step, which is of course not possible.
    Again, I think it's impossible to do better. It was certainly fast so I would have to stay very concentrated, but since it was a hard course it was expected.
    Sometimes it felt a bit slower in the beginning and too rushed in the end. But it might be that I got more tired.
    As the lectures often were not finished during the lecture time, one interpretation would be that they were too slow, but they contained a lot of information which could probably not be processed any quicker. When trying to rush in order to get through everything, the pace goes too high.

  15. Some lectures had slides while most were given on the board. What do you think about the balance between slides and on-the-board presentations?

    1. 13% (1 st) All lectures should be on the board.
    2. 50% (4 st) Most lectures should be on the board (as was the case now).
    3. 0% (0 st) Half of the lectures should be on the board and half should have slides.
    4. 0% (0 st) Most lectures should be with slides.
    5. 0% (0 st) All lectures should be with slides.
    6. 25% (2 st) No particular views on this one.

    Any comments about slides versus board presentations?

    an advantage of slides is that they can be reused as recap material for the students later
    Slides are OK when the lecture is about introducing many not so complicated concepts (e.g. introduction lecture) but it is in my opinion hard to follow a proper proof presented this way.
    Yes, the material is mostly suited for the board I think.
    Given that this is a small course, making slides is too much work with little gain. I don't see how slides are better, perhaps only in that you can view them online afterwards.
    If slides are used, they are their own lecture notes (the non-scribed version, I mean). If the lecture happens on the board, there is time to take notes. If slides are used, but some things are explained on the board, the class would benefit from having the slides printed so that notes about the board explanation goes into context. The same goes for extra slides that are not used, or actually arenīt part of the explanation of the lecture - these slides could easier be marked as "not relevant".
    Also, the scanned lecture notes were helpful when understanding what the teacher wanted to explain - such traces were not available when slides was all we had. If something, for instance an illustration, is easier to perform on slides, slides should be used. When "building up" the mathematical setting for something, slides are sometimes even confusing the story, by presenting too much at the same time (and not showing traces of what happend in what order, if processes are flattened).

    I generally prefer board for mathematics. Slides are ok for surveys.

  16. Did you prepare for the lectures in advance (by looking through the handwritten lecture notes, say, or refreshing your memory about what was said last time, or by looking at the paper(s) to be covered during the lecture?)

    1. 0% (0 st) Yes, always.
    2. 13% (1 st) Yes, often.
    3. 38% (3 st) Sometimes.
    4. 38% (3 st) Seldom.
    5. 13% (1 st) Pretty much never.
    6. 0% (0 st) Not applicable.

  17. The course had a fair amount of guest lectures. What do you think about having guest lectures?

    1. 25% (2 st) Very good — they should definitely be kept.
    2. 38% (3 st) Good — it's reasonably valuable to have them.
    3. 25% (2 st) Neutral — they didn't hurt but skipping them would probably be just as fine.
    4. 13% (1 st) Fairly bad — we should have had less of them.
    5. 0% (0 st) Very bad — guest lectures should have been skipped altogether.
    6. 0% (0 st) Don't know — didn't attend the guest lectures.

    Any comments or feed-back about the guest lectures in general?

    I very much enjoyed them!
    It's a good idea, but they were of varying quality.
    They were all different. Differently experienced speakers, I guess. It is no good if the majority of the class only have access to slides about something similar to the lecture (but not the lecture itself), for a long time.
    The guest lectures were nice, but naturally not as well integrated with the rest of the course as the normal lectures. In particular, the guest lectures did not know exactly what we had and had not covered before. Overall, it was nice seeing what other people worked on, but the time was not spent as efficiently as the normal lectures from a learning point of view.

  18. How did you like Matti Järvisalo's guest lecture on SAT solving?

    1. 63% (5 st) Very good.
    2. 25% (2 st) Good.
    3. 0% (0 st) Acceptable.
    4. 0% (0 st) Fairly bad.
    5. 0% (0 st) Very bad.
    6. 0% (0 st) Don't know — didn't attend the lecture.

    Any comments or feed-back to Matti?

    It was interesting to see how minisat, a program with used a lot in the assignments, worked. It is also always nice to see the "real life applications" of what we study, even in a theoretical course.
    Very interesting. If I remember it correctly he ran out of time so we had to do an extra unofficial lecture. Both were very enjoyable!
    From what I remember, it was very good. He explained a reasonable subset of the topic in reasonable pace and with well chosen examples.

  19. How did you like Chris Beck's guest lectures on time-space trade-offs?

    1. 13% (1 st) Very good.
    2. 25% (2 st) Good.
    3. 38% (3 st) Acceptable.
    4. 13% (1 st) Fairly bad.
    5. 0% (0 st) Very bad.
    6. 0% (0 st) Don't know — didn't attend the lectures.

    Any comments or feed-back to Chris?

    To be honest the speed with which Chris talked was a problem for me, but nothing too bad. It was nice to hear him talking about a not-even-published-yet paper.
    He talks really really fast, but he also has a great amount of energy and enthusiasm. The material he presented was incredibly interesting!
    He spoke too fast and the problem was also very hard. It was hard to see the "big picture in his lecture".
    Hi was rather confusing to listen to, because he juggled about the topic and jumped between different parts without clearly stating what he considered he was trying to explain. It is no good if the students need to use most of their capacity to figure out "what part of the topic could be explained by reasoning about what the lecturer is currently reasoning about". The result was so new, and the approach jumped between the old methods and the new ones. He was eager to explain well, that showed, but it is difficult to pose question before understanding what it is you want to know.

  20. How did you like Samuel Lundqvist's guest lecture on algebraic SAT solving?

    1. 0% (0 st) Very good.
    2. 38% (3 st) Good.
    3. 25% (2 st) Acceptable.
    4. 13% (1 st) Fairly bad.
    5. 0% (0 st) Very bad.
    6. 13% (1 st) Don't know — didn't attend the lecture.

    Any comments or feed-back to Samuel?

    The subject was interesting but too much time was spent "introducing" already known algebraic concepts.
    What he talked about the first hour we already had pretty much covered (in a slightly different fashion) in the main lectures. But it was a seminar open to the whole of CSC so I understand why he couldn't really make that part briefer. The material was interesting and I would have enjoyed some more coverage of how the frontier looks there at the moment. Maybe something about current implementations, how they are doing in comprehension to SAT solvers based on Resolution and some guesses of why.
    A bit simple.
    He did not reason above our heads, at least. But he explained some things very much at length.

  21. What do you think about the connections made to applications (SAT solving) during the course? How much such connections should have been made?

    1. 13% (1 st) Way more connections.
    2. 13% (1 st) A bit more.
    3. 63% (5 st) About right as it was.
    4. 0% (0 st) A bit less.
    5. 13% (1 st) Way less connections.
    6. 0% (0 st) No opinion.

    Scribe notes

  22. All students on the course were requested to scribe two lectures in LaTeX. How useful for you personally was it to scribe lectures (asking not about the end result, but about the process)?

    1. 38% (3 st) Very useful.
    2. 50% (4 st) Fairly useful.
    3. 13% (1 st) Neutral.
    4. 0% (0 st) Not very useful.
    5. 0% (0 st) Not at all useful.
    6. 0% (0 st) Don't know — didn't do scribing.

  23. How useful was it to have access to and be able to study the scribed lecture notes produced?

    1. 75% (6 st) Very useful.
    2. 25% (2 st) Fairly useful.
    3. 0% (0 st) Neutral.
    4. 0% (0 st) Not very useful.
    5. 0% (0 st) Not at all useful.
    6. 0% (0 st) Don't know — didn't read scribe notes.

    Do you have any comments or feed-back regarding scribing and/or the scribe notes?

    I think from a methodological perspective, that is to learn how to scribe notes, I do not feel that I would have needed this exercise. However, it helped quite much in understanding the content of the lecture in question. But then again, I am wondering if it is not better to use this time (it takes quite some time to scribe) for learning about ALL lectures content in some other way.

    It was very useful/helpful/important to have access to lecture notes. And having this in a nice scribed PDF with full text etc. is of course the best you can get, but I guess, just the scanned copies of the lecturers handwritten notes (or used slides) would have been fine as well.

    I must highlight how useful these were. First, they were of course providing very good information when solving the problem sets. But writing them was also a very good exercise! I was actually able to appreciate the skills I gained in doing it when writing assignments for other courses.
    The scribing was on the brink of being a bit too hard. Jakob has very high standards and here I felt was the most difficult part of living up to them. I am a bit conflicted of my own opinion here...

    It will be noted, however, that the published notes was very nice as reading material since they pretty much felt like a good university textbook because of the high quality.

    The original deadline for scribing was way to short. Since notes were usually not finished by this deadline, it moved, and for the process of scribing this is good. For the purpose of solving homeworks, however, it is bad when notes donīt even appear during the course. I donīt know how to solve this issue, maybe if there are always scanned lecture notes and slides that are up to date, late scribed notes can be allowed? Now sometimes, no notes at all were available.

    Problem sets

  24. Did you find the problem sets easy or hard?

    1. 0% (0 st) Very easy.
    2. 0% (0 st) Fairly easy.
    3. 0% (0 st) Medium.
    4. 63% (5 st) Fairly hard.
    5. 38% (3 st) Very hard.
    6. 0% (0 st) Don't know — didn't do problem sets.

  25. What do you think about the level of difficulty of the problem sets?

    1. 0% (0 st) Far too easy.
    2. 0% (0 st) A bit too easy.
    3. 63% (5 st) About right.
    4. 25% (2 st) A bit too hard.
    5. 13% (1 st) Far too hard.
    6. 0% (0 st) Don't know — didn't do problem sets.

  26. What do you think about the number of problem sets and the number of problems per set?

    1. 0% (0 st) Far too few problems all in all.
    2. 0% (0 st) A bit too few problems.
    3. 63% (5 st) About right.
    4. 25% (2 st) A bit too many.
    5. 13% (1 st) Far too many.
    6. 0% (0 st) Don't know — didn't do problem sets.

  27. What do you think about the deadlines for the problem sets and how much time was given for them?

    1. 0% (0 st) Far too much time.
    2. 0% (0 st) A bit too much time.
    3. 38% (3 st) About right.
    4. 50% (4 st) A bit too little time.
    5. 13% (1 st) Far too little time.
    6. 0% (0 st) Don't know — didn't do problem sets.

  28. What do you think about the grading of the problem sets (based on the pset solutions returned to you so far)?

    1. 0% (0 st) Far too harsh.
    2. 13% (1 st) A bit too harsh.
    3. 75% (6 st) About right.
    4. 0% (0 st) A bit too lenient.
    5. 0% (0 st) Far too lenient.
    6. 13% (1 st) Not applicable.

  29. What do you think about the balance between theoretical and applied problems (involving coding) on the psets?

    1. 0% (0 st) Far too many theoretical problems.
    2. 13% (1 st) A bit too many theoretical problems.
    3. 75% (6 st) About the right balance.
    4. 13% (1 st) A bit too many applied problems.
    5. 0% (0 st) Far too many applied problems.
    6. 0% (0 st) Don't know — didn't do problem sets.

  30. How much time on average would you say that you spent per problem set?

    1. 0% (0 st) Less than 5 hours.
    2. 0% (0 st) 5-10 hours.
    3. 0% (0 st) 10-20 hours (up to half a week of working time).
    4. 25% (2 st) 20-30 hours.
    5. 50% (4 st) 30-40 hours (up to a full week of working time).
    6. 25% (2 st) More than 40 hours.
    7. 0% (0 st) Not applicable.

  31. How well do you think the problem sets corresponded to the material presented during the lectures?

    1. 50% (4 st) Very well.
    2. 50% (4 st) Fairly well.
    3. 0% (0 st) Somewhat, but not too well.
    4. 0% (0 st) Fairly badly.
    5. 0% (0 st) Very badly.
    6. 0% (0 st) Don't know — didn't do problem sets.

    Do you have any comments or feed-back on the problem sets?

    Maybe the first problem set could come earlier within the course? That might help in spreading the problem set more evenly throughout time.
    The problems in the problem sets themselves were really really good (most of them anyway). I would've had prefered to have the first PS split into two sets of smaller size and the first one could be given earlier. I recalled actually being somewhat shocked of the size of the first PS when I looked at it :)

    I strongly disliked the overlapping deadlines for some of the problem sets. That is, when a new PS has been posted without the previous one having it's deadline passed.

    But I really like to emphasize that Jakob did a good work on coming up with problems that were both connected to the course and not too hard nor too easy.

    The problem sets were great, but given that we scribed notes as well, there were simply too many.
    The practical rules about problem sets felt a bit rigid. What could have happened if you were allowed to discuss problems with as many people of the course as you wanted to, but had to acknowledge this? Would people really have done less work? Understood less? Or maybe actually understood more? The hints were a difficult subject too, although the idea was good. In general, you wouldnīt ask for a hint before trying to solve the problem, but as soon as some time has been invested in trying, you risk that the hint is only what you have been able to deduce so far, and not really something that directs you anywhere from there.
    The problem sets were rather difficult and time consuming, but I think this is appropriate for a large grad level course.

    I was not able to complete all problem sets on time due to other engagements, and so to me it was far too little time. On the other hand, the allocated time seem about right assuming I had the time to really follow the course.

    Workload and course credits

  32. How much of your total study/working time did you spend on this course (on average per week, say)?

    1. 0% (0 st) Less than 15%.
    2. 13% (1 st) 15-30%.
    3. 88% (7 st) 30-50%.
    4. 0% (0 st) 50-70%.
    5. 0% (0 st) More than 70%.

  33. This course gives 9 ECTS credits. What you you think about this compared to the number of credits given for other courses?

    1. 63% (5 st) Should have given more than 9 credits.
    2. 38% (3 st) 9 credits was about right.
    3. 0% (0 st) Should have given less credits.

    Concluding questions

  34. Did you experience that you were discriminated against on this course due to gender, sexuality, ethnicity, or disability?

    1. 0% (0 st) Yes.
    2. 0% (0 st) Don't know.
    3. 100% (8 st) No.

    If yes, in which way?

  35. How do you assess the course from a gender perspective (e.g., with respect to course material, course contents, lecturer et cetera)?

    It had about the right amount of gender perspective...
    As usual, I donīt think this applies to mathematical courses unless the lecturer would actually behave insinuant or hostile to people, or make claims or comments, or hint premises, which sort and value people by gender. He did not. I donīt know what a gender perspective on proofs is.

  36. Do you think that a course like this should be given on a regular basis (say, every second or third year or so) at KTH?

    1. 38% (3 st) Yes, definitely.
    2. 25% (2 st) Yes, probably.
    3. 25% (2 st) Neutral, could argue both ways.
    4. 13% (1 st) No, probably not.
    5. 0% (0 st) No, definitely not.
    6. 0% (0 st) No opinion.

    Would you like to elaborate on your answer?

    topics are very interesting, most lectures are well organized. also important to learn new ideas from other areas.
    It actually depends on the research interests of the students. I think for students working in this area, it is definitely worth attending the course. However, as for me personally, I have chosen the course as I thought it might be interesting even though it is not related to my research. It turned out that it was indeed interesting, but it took much more effort and time and caused much more worries and sorrow than anticipated. Of course, courses are supposed to be a challenge for the student, but after all there are many other duties for a research student and I had to turn down quite some wishes from my supervisors and could not really proceed my own research to such an extend as hoped for while I was taking the course. So that all in all, to be honest, I would probably recommend other students to invest their time and energy into producing more research results and papers or cover a broader variety of knowledge by choosing two smaller courses or such like instead.
    I mean, as for variety, the course goes very much into detail, which is fine per se, but does not buy the student so much for his/her later work unless he/she is going to work in exactly that field. If, in contrast, the student is taking courses from other research areas in order to look beyond his/her own nose, then he/she can get an introduction to many more fields with picking other or several small courses which sum up to 9 credits.

    Due to its very close relation with both SAT-solving and complexity theory, I think such a course *must* be given on a regular basis. Maybe not every year (though that would prevent some exchange students from taking it) but definitely every second year.
    Although it is a hard course, it is very rich in "scientific content". I personally enjoyed the course very much although I had to work a lot harder for it than an ordinary course. I think if it is given again, the reputation of what is required will be spread and so the students will be more prepared for it, mentally.

    Jakob really really knows his field, so he has an almost unique ability to explain the material thus it would be a shame if it was not given again.

    Too specialized.
    I donīt exaclty know what courses are given regularly, but the content of this course is probably valid (although less new) in a couple of years. Probably, some introductions are needed for all research topics here, and a course is a nice introduction.
    I appreciate that people are giving research level courses. A greater number of courses to choose from would be helpful. On the other hand, since this course is directed primarily at PhD students, I don't think it is practical to give the course often as the number of new PhD students each year is fairly small. Perhaps one could have some rolling schedule with this course, Johan's toolkit course and 1 or 2 other courses to ensure availability of courses without giving the same course multiple times during the studies of each grad student.

  37. Do you have any comments on aspects of the course that should stay the same in a possible next round of the course?

    most parts --- form of lectures, problem sets, scribe notes.
    The examination in the form of problem sets scribing was good. As mentioned before, the scribing was possibly a bit too hard, but the problem sets were really good.
    Lectures and the problems in problem sets.
    Using homeworks and scribing as course requirements is good. Lectures are also good, since reading the papers does not put things into context as well as the lecturer does.

  38. Do you have any suggestions for how the course could be improved?

    some questions in the problem sets are a bit repetitive. balance between programming and other questions.
    I think this course would benefit from a rise of its credits, maybe up to 12, especially for master's students.
    As mentioned earlier, a smaller first problem set would be nice.
    Fewer problem sets. Be less ambitious about covering a lot of material and try to rush less towards the end of the lecture.
    Scribing lecture notes, at least for guest lectures, takes _a lot_ of time. More than half a problem set. It should be worth more. Maybe more scribing, but in pairs, could be a solution? Two people collect thoughts during the lecture, sketch up the arguments independently, and cooperate to fill in the blanks?
    Maybe change the regulations about cooperation to prohibit whatever it is supposed to prohibit by stipulating groups of size at most three, more explicitly, and have the students explain how they have followed this code of conduct.

  39. Any other final comments on the course?

    This course is fascinating but very demanding. It takes nowhere near 50% more time than a 6 ECTS course to pass it so I think it would benefit from a modification, not from the content (fine as it is in my opinion) but rather from an augmentation of the credits given (at least for master's student). Besides, it uses lots of different tools, from graph theory and complexity theory to polynomial ring algebra so I really think it would be worth it.
    One of the very best and interesting courses I have taken here at KTH!

These results were compiled with ACE.