Overview Challenge Guidelines

Real Open-Ended

Standard  Dev  Challenge


Results as of 05/07/2021 (deadline for VQA Challenge 2021).
For information about each test split, please see the challenge page.

By Answer Type
Overall
Yes/No
Number
Other

Human Performance

95.49

80.84

67.89

80.78

Renaissance

93.27

68.4

70.65

79.78

PASH-SFE

92.56

74.85

67.67

78.8

UNIMO

93.26

63.99

69.27

78.63

ROSITA

92.6

63.76

69.42

78.4

sunlan

92.58

67.52

67.59

77.95

Summer

92.55

64.75

67.7

77.68

Katya

92.08

61.57

67.03

76.81

TaiwanTech

91.95

61.01

67.18

76.76

HSSLAB_INSPUR

92.13

61.85

66.65

76.68

Modal Master

92

62.25

66.6

76.65

MIL-nbgao

90.6

58.42

66.47

75.58

dice

91.28

57.78

65.77

75.46

test987456

91.09

56.95

65.58

75.19

liuyuhang

89.87

58.68

64.97

74.59

Harold

89.64

53.48

65.61

74.22

Uniter-CL

89.8

55.98

62.97

73.31

threepigs

88.47

55.64

63.93

73.18

XMU_MAC_VQA

88.37

55.57

63.51

72.93

zxxw

87.98

54.89

63.44

72.66

lisizhandui

87.84

54.46

63.27

72.47

test_tju

87.85

54.75

62.7

72.24

snow

87.78

53.49

62.63

72.04

zxs

87.79

53.49

62.63

72.04

Haris

87.58

55.16

62.08

71.88

enenenne

87.1

55.96

61.78

71.63

vqa

87.23

55.99

61.47

71.54

outlook_vqa

87.53

53.56

61.69

71.5

Moka

87.52

52.86

61.73

71.44

vqa_ccq

87.3

53.64

61.66

71.4

cp

87.41

53.35

61.59

71.38

vqa2

87.33

52.98

61.7

71.36

HooLoo

87.24

53.22

61.57

71.29

YichaoWang

87.26

52.75

61.67

71.29

vqa1

87.23

52.84

61.69

71.29

lpp

87.23

53.69

61.28

71.19

shi

87.15

53.43

61.2

71.1

hhhy

87.07

52.52

61.03

70.88

tomato

87.04

51.24

61.3

70.85

Ana

86.82

53.9

60.79

70.82

PHIKN1GHT

87.35

51.46

60.87

70.8

Hao Shen

86.62

51.57

60.9

70.52

Telsa2

86.56

52.64

60.4

70.38

Telsa

86.49

52.52

60.41

70.34

RWSAN

86.28

51.59

60.78

70.33

trouble

86.51

51.5

59.74

69.92

ckjzjh

85.56

53.02

59.47

69.57

muziyongshixin

85.66

50.34

60.02

69.57

MC-CAN

85.37

49.28

58.22

68.48

Arthurs

83.39

48.95

59.14

68.06

gxnu

82.31

46.37

55.71

65.7

AL_polaris

79.29

43.48

53.16

62.92

Bazavan Cristian

68.61

31.67

30.34

46.36

AI beast from mcgill

65.44

27.38

18.7

39.06

SteveM Lancaster Uni

52.9

23.89

9.33

29.03

mafia

0

0

0.05

0.02

Please see the evaluation page for information about the metrics.