Quality criteria for sociology? What sociologists can learn from the project «developing and testing research quality criteria in the humanities»

Journal title SOCIOLOGIA E POLITICHE SOCIALI
Author/s Michael Ochsner, Tobias Wolbring, Sven E. Hug
Publishing Year 2015 Issue 2015/2
Language English Pages 21 P. 90-110 File size 126 KB
DOI 10.3280/SP2015-002005
DOI is like a bar code for intellectual property: to have more infomation click here

Below, you can see the article first page

If you want to buy this article in PDF format, you can do it, following the instructions to buy download credits

Article preview

FrancoAngeli is member of Publishers International Linking Association, Inc (PILA), a not-for-profit association which run the CrossRef service enabling links to and from online scholarly content.

Universities take an important role in the knowledge-society. For reasons of accountability to the public or in order to assure or enhance research quality, many universities implemented assessment procedures, often using bibliometric and other performance indicators. These procedures are mostly developed in a data-driven manner and not much is known about what the indicators in these procedures actually measure and how they affect behavior. Furthermore, the methods stem from the natural and life sciences and cannot be readily transferred to the social sciences and humanities. In this article, we present (i) quality criteria for research from the perspective of humanities scholars and how they can be transferred to sociology (ii) summarise the opportunities and limitations of the research rating of the German Council of Science and Humanities, and (iii) suggest that sociology as a discipline should develop a discipline-specific approach to research evaluation that takes into account the sociology scholars’ notions of quality and the disciplines’ research practices, that is bottom-up in nature, and uses both quantitative as well as qualitative data.

Keywords: Research Evaluation; Quality Criteria; Sociology; Research Policy; Research Assessment

  1. Alexander, F. K. 2000. The Changing Face of Accountability: Monitoring and Assessing Institutional Performance in Higher Education. Journal of Higher Education, 71(4), pp. 411–431. DOI: 10.2307/264914
  2. Andersen, H., Ariew, R., Feingold, M., Bag, A. K., Barrow-Green, J., van Dalen, B., Zuidervaart, H. (2009). Editorial: Journals under Threat: A Joint Response from History of Science, Technology and Medicine Editors, Social Studies of Science, 39(1), pp. 6-9. DOI: 10.1177/0306312709039001070
  3. Auspurg, K., Diekmann, A., Hinz, T., Naef, T. 2015. Das Forschungsrating des Wissenschaftsrats für die Soziologie in Deutschland revisited. [The Research Rating of the German Council of Science and Humanities on Sociology Revisited], Soziale Welt, 66(3), forthcoming.
  4. Beck, U. 1992. Risk Society. Towards a New Modernity. London: Sage. Bell, D. 1973. The Coming of Post-industrial Society: A Venture in Social Forecasting. New York: Basic Books.
  5. Butler, L. (2003). Modifying Publication Practices in Response to Funding Formulas, Research Evaluation, 17(1), pp. 39-46.
  6. The Climate Group. 2007, June 1st. President Bush Sends Mixed Signals on Climate Treaty ahead of G8. The Climate Group News Blog. Retrieved from http://www.theclimategroup.org/what-we-do/news-and-blogs/President-Bushsends-mixed-signals-on-climate-treaty-ahead-of-G8/
  7. Wissenschaftsrat. 2004. Empfehlungen zu Rankings im Wissenschaftssystem. Teil I: Forschung. Köln.
  8. Dávidházi, P. 2014. Exploring Paradigms and Ourselves. In P. Dávidházi, ed. New Publication Cultures in the Humanities. Exploring the Paradigm Shift, pp. 9-18. Amsterdam: Amsterdam University Press.
  9. Delbecq, A. L., Van de Ven, A., and Gustafson, D. H. 1975. Group Techniques for Programm Planning. A Guide to Nominal Group and Delphi Processes. Glenview: Scott, Foresman.
  10. Donovan, C. 2007. The Qualitative Future of Research Evaluation. Science and Public Policy, 34(8), pp. 585-597. DOI: 10.3152/030234207X25653
  11. Fisher, D., Rubenson, K., Rockwell, K., Grosjean, G., and Atkinson-Grosjean, J. 2000. Performance Indicators and the Humanities and Social Sciences. Vancouver: Centre for Policy Studies in Higher Education and Training, University of British Columbia.
  12. Geschwind, L., and Larsson, K. 2008. Getting Pole Position - Pre Reform Research Strategies in the Humanities at Swedish Universities. Working Paper Series in Economics and Institutions of Innovation No. 140. The Royal Institute of Technology.
  13. Gläser, J., Laudel, G., Hinze, S., and Butler, L. 2002. Impact of Evaluation-based Funding on the Production of Scientific Knowledge: What to Worry About, and How to Find Out. Expertise for the German Ministry for Education and Research.
  14. Hayek, F. 1937. Economics and Knowledge. Economica, 4, pp. 33-54.
  15. Herbert, U., and Kaube, J. 2008. Die Mühen der Ebene: Über Standards, Leistung und Hochschulreform. In E. Lack and C. Markschies, eds., What the hell is quality? Qualitätsstandards in den Geisteswissenschaften, pp. 37-51. Frankfurt: Campus.
  16. Hicks, D. 2004. The Four Literatures of Social Science. In H. F. Moed, W. Glänzel and U. Schmoch, eds., Handbook of Quantitative Science and Technology Research: The Use of Publication and Patent Statistics in Studies of S&T Systems, pp. 473-496. Dordrecht: Kluwer Academic.
  17. Hug, S. E., Ochsner, M., and Daniel, H.-D. 2013. Criteria for Assessing Research Quality in the Humanities – A Delphi Study Among Scholars of English Literature, German Literature and Art History. Research Evaluation, 22(5), pp. 369-383. DOI: 10.1093/reseval/rvt00
  18. — 2014. A Framework to Explore and Develop Criteria for Assessing Research Quality in the Humanities. International Journal for Education Law and
  19. Policy, 10(1), pp. 55-64. Jankowiecz, D. 2001. Why Does Subjectivity Make Us Nervous? Making the Tacit Explicit. Journal of Intellectual Capital, 2(1), pp. 61-73. DOI: 10.1108/1469193011038050
  20. Kelly, G. A. 1955. The Psychology of Personal Constructs. New York: Norton.
  21. Lack, E. 2008. Einleitung – das Zauberwort “Standards”. In E. Lack and Markschies, C., eds. What the Hell Is Quality? Qualitätsstandards in den Geisteswissenschaften, pp. 9-34. Frankfurt a.M.: Campus.
  22. Lane, R. E. 1966. The Decline of Politics and Ideology in a Knowledgeable Society. American Sociological Review, 31, pp. 649–662.
  23. Lariviere, V., Gingras, Y., and Archambault, E. 2006. Canadian Collaboration Networks: A Comparative Analysis of the Natural Sciences, Social Sciences and the Humanities. Scientometrics, 68(3), pp. 519-533.
  24. Lawrence, P. A. 2003. The Politics of Publication. Authors, Reviewers and Editors Must Act to Protect the Quality of Research. Nature, 422, pp. 259-261.
  25. Linstone, H. A., and Turoff, M. 1975. The Delphi Method: Techniques and Applications. Reading, MA: Addison-Wesley.
  26. Mora, J.-G. 2001. Governance and Management in the New University. Tertiary Education and Management, 7(2), pp. 95-110. DOI: 10.1023/A:101133801608
  27. Münch, R. 2009. Die Konstruktion soziologischer Exzellenz durch Forschungsrating [The Construction of Sociological Excellence through Research Rating.]. Soziale Welt 60(1): 63-89.
  28. Nederhof, A. J. 2006. Bibliometric Monitoring of Research Performance in the Social Sciences and the Humanities: A review. Scientometrics, 66(1), pp. 81-100.
  29. Neidhardt, F. 2006. Forschungsrating der deutschen Soziologie durch den Wissenschaftsrat. Soziologie, 35, pp. 303-308.
  30. — 2008. Das Forschungsrating des Wissenschaftsrats. Einige Erfahrungen und Berichte. Soziologie, 37, pp. 421-432.
  31. — 2009. Über Nachteile von Vorteilen. Ein Kommentar zu Richard Münch: „Die Konstruktion soziologischer Exzellenz durch Forschungsrating“, in: Soziale Welt 60: 63-89. Soziale Welt, 60(3), pp. 325-333.
  32. Nussbaum, M. C. 2010. Not for Profit. Why Democracy Needs the Humanities. Princeton, NJ: Princeton University Press.
  33. Ochsner, M., Hug, S. E., and Daniel, H.-D. 2012. Indicators for Research Quality in the Humanities: Opportunities and Limitations. Bibliometrie - Praxis und Forschung, 1(4). URN: urn:nbn:de:bvb:355-157-7.
  34. — 2013. Four Types of Research in the Humanities: Setting the Stage for Research Quality Criteria in the Humanities. Research Evaluation, 22(2), pp. 79-92. DOI: 10.1093/reseval/rvs03
  35. — 2014. Setting the Stage for the Assessment of Research Quality in the Humanities: Consolidating the Results of Four Empirical Studies. Zeitschrift für Erziehungswissenschaften, 17(Supplement 6), pp. 111–132. DOI: 10.1007/s11618-014-0576-
  36. Peric, B., Ochsner, M., Hug, S. E., and Daniel, H.-D. 2012. AHRABi. Arts and Humanities Research Assessment Bibliography. ETH Zurich. Peters, M. A. 2007. Knowledge Economy, Development and the Future of Higher Education. Rotterdam: Sense Publishers.
  37. Plumpe, W. 2009. Stellungnahme zum Rating des Wissenschaftsrates aus Sicht des Historikerverbandes. In C. Prinz and R. Hohls, eds., Qualitätsmessung,
  38. Evaluation, Forschungsrating. Risiken und Chancen für die Geschichtswissenschaften?, pp. 121–126. Historisches Forum. Berlin: Clio-online.
  39. Readings, B. 1996. The University in Ruins. Cambridge, MA: Harvard University Press.
  40. Riordan, P., Ganser, C., and Wolbring, T. 2011. Zur Messung von Forschungsqualität. Eine kritische Analyse des Forschungsratings des Wissenschaftsrats [Measuring the quality of research – A critical analysis of the Forschungsrating of the German Wissenschaftsrat]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 63(1), pp. 147-172.
  41. Rolfe, G. 2013. The University in Dissent. Scholarship in the Corporate University. Abingdon: Routledge.
  42. Stehr, N. 1994. Knowledge Societies. London: Sage.
  43. Thorngate, W., Dawes, R. M., and Foddy, M. 2009. Judging Merit. New York, NY: Psychology Press. UNESCO 1998. World Declaration on Higher Education for the Twenty-first Century: Vision and Action. And: Framework for Priority Action for Change and Development in Higher Education. Paris: UNESCO.
  44. — 2005. Toward knowledge societies. UNESCO World Report. Paris: UNESCO. Välimaa, J. and Hoffman, D. 2008. Knowledge Society Discourse and Higher Education. Higher Education, 56(3), pp. 265–285. DOI: 10.1007/s10734-008-9123-
  45. Weingart, P. 2008. Was ist gesellschaftlich relevante Wissenschaft? In A. Schavan, ed., Keine Wissenschaft für sich. Essays zur gesellschaftlichen Relevanz von Forschung, pp. 15-24. Hamburg: Edition Körber Stiftung.
  46. Weingart, P. 2015. Nostalgia for the World without Numbers. Soziale Welt, 66(3), forthcoming.
  47. — 2008a. Pilotstudie Forschungsrating. Empfehlungen und Dokumentation. Köln.
  48. — 2008b. Forschungsleistungen deutscher Universitäten und außeruniversitärer Einrichtungen in der Soziologie. Köln.

  • The future of research assessment in the humanities: bottom-up assessment procedures Michael Ochsner, Sven Hug, Ioana Galleron, in Palgrave Communications 17020/2017
    DOI: 10.1057/palcomms.2017.20

Michael Ochsner, Tobias Wolbring, Sven E. Hug, Quality criteria for sociology? What sociologists can learn from the project «developing and testing research quality criteria in the humanities» in "SOCIOLOGIA E POLITICHE SOCIALI" 2/2015, pp 90-110, DOI: 10.3280/SP2015-002005