WEBVTT 00:00:00.000 --> 00:00:01.853 align:middle line:90% 00:00:01.853 --> 00:00:03.270 align:middle line:84% BRIANNA BLASER: Well great, folks. 00:00:03.270 --> 00:00:05.243 align:middle line:84% I think we'll go ahead and get started. 00:00:05.243 --> 00:00:06.660 align:middle line:84% For those of you who I don't know, 00:00:06.660 --> 00:00:09.210 align:middle line:84% my name's Brianna Blaser, with AccessCSforAll 00:00:09.210 --> 00:00:11.360 align:middle line:90% at the University of Washington. 00:00:11.360 --> 00:00:14.460 align:middle line:84% And as you may know, we work to help 00:00:14.460 --> 00:00:16.050 align:middle line:84% ensure that K-12 computer science 00:00:16.050 --> 00:00:18.600 align:middle line:90% education is accessible. 00:00:18.600 --> 00:00:21.780 align:middle line:84% And we are always happy to work with Andreas Stefik 00:00:21.780 --> 00:00:25.200 align:middle line:84% from the University of Nevada, Las Vegas. 00:00:25.200 --> 00:00:28.230 align:middle line:84% And so today, Stefik is going to share with us 00:00:28.230 --> 00:00:32.590 align:middle line:84% some of the work that he's been doing related to data science. 00:00:32.590 --> 00:00:34.680 align:middle line:84% So I'm going to turn it over to Stefik 00:00:34.680 --> 00:00:36.167 align:middle line:90% and let him take it from here. 00:00:36.167 --> 00:00:37.390 align:middle line:90% ANDREAS STEFIK: All right. 00:00:37.390 --> 00:00:40.685 align:middle line:90% Well, thank you, Brianna. 00:00:40.685 --> 00:00:43.060 align:middle line:84% And it looks like I actually know quite a few people that 00:00:43.060 --> 00:00:43.740 align:middle line:90% showed up. 00:00:43.740 --> 00:00:45.960 align:middle line:90% Neil Brown, I see. 00:00:45.960 --> 00:00:47.670 align:middle line:84% It's got to be like 2:00 AM there. 00:00:47.670 --> 00:00:48.540 align:middle line:90% I don't even know. 00:00:48.540 --> 00:00:49.090 align:middle line:90% So cool. 00:00:49.090 --> 00:00:52.680 align:middle line:90% Well, thank you all for coming. 00:00:52.680 --> 00:00:54.730 align:middle line:84% Brianna, can you confirm that my screen 00:00:54.730 --> 00:00:56.207 align:middle line:90% is showing on the share screen? 00:00:56.207 --> 00:00:57.040 align:middle line:90% BRIANNA BLASER: Yep. 00:00:57.040 --> 00:00:57.540 align:middle line:90% Absolutely. 00:00:57.540 --> 00:00:58.098 align:middle line:90% It's there. 00:00:58.098 --> 00:00:58.890 align:middle line:90% ANDREAS STEFIK: OK. 00:00:58.890 --> 00:01:00.310 align:middle line:90% Cool. 00:01:00.310 --> 00:01:02.140 align:middle line:90% So, yeah. 00:01:02.140 --> 00:01:02.740 align:middle line:90% Andy Stefik. 00:01:02.740 --> 00:01:04.150 align:middle line:90% Everybody calls me Stefik. 00:01:04.150 --> 00:01:10.030 align:middle line:84% I work in Vegas and I'm one of the PIs on AccessCSforAll. 00:01:10.030 --> 00:01:12.190 align:middle line:84% We do a lot of accessibility work in high school. 00:01:12.190 --> 00:01:14.200 align:middle line:84% If you don't know the kind of stuff, 00:01:14.200 --> 00:01:17.620 align:middle line:84% the other PI on the project is Richard Ladner out 00:01:17.620 --> 00:01:19.210 align:middle line:90% of University of Washington. 00:01:19.210 --> 00:01:22.475 align:middle line:90% Also Sheryl Burgstahler as well. 00:01:22.475 --> 00:01:24.100 align:middle line:84% So what we're going to talk about today 00:01:24.100 --> 00:01:27.970 align:middle line:84% is some work that came about because of some changes that 00:01:27.970 --> 00:01:29.910 align:middle line:90% were made to the-- 00:01:29.910 --> 00:01:31.660 align:middle line:84% in the United States, there is this entity 00:01:31.660 --> 00:01:33.370 align:middle line:90% called the college board. 00:01:33.370 --> 00:01:37.570 align:middle line:84% And they basically control classes in high school, 00:01:37.570 --> 00:01:38.890 align:middle line:90% effectively. 00:01:38.890 --> 00:01:40.930 align:middle line:84% And one of the units that they added 00:01:40.930 --> 00:01:44.690 align:middle line:84% was related to data which caused the organization code.org 00:01:44.690 --> 00:01:46.420 align:middle line:90% to create these data units. 00:01:46.420 --> 00:01:48.730 align:middle line:84% And then the question became, well how exactly do you 00:01:48.730 --> 00:01:51.640 align:middle line:84% handle this kind of data or data science type applications 00:01:51.640 --> 00:01:53.660 align:middle line:90% in an accessible way? 00:01:53.660 --> 00:01:55.480 align:middle line:84% Also this led to some other conversations 00:01:55.480 --> 00:01:58.390 align:middle line:84% with Gary Ritchie, who is also on the call, 00:01:58.390 --> 00:02:01.090 align:middle line:84% related to some work that are doing about accessibility 00:02:01.090 --> 00:02:05.390 align:middle line:90% with data science in R as well. 00:02:05.390 --> 00:02:07.170 align:middle line:90% So will this actually go? 00:02:07.170 --> 00:02:08.830 align:middle line:90% There it is. 00:02:08.830 --> 00:02:11.800 align:middle line:84% So I want to give just a little bit of fair warning 00:02:11.800 --> 00:02:12.730 align:middle line:90% on this talk. 00:02:12.730 --> 00:02:14.915 align:middle line:84% It's maybe not a good idea to give fair warning 00:02:14.915 --> 00:02:16.540 align:middle line:84% and then watch all the people drop off, 00:02:16.540 --> 00:02:18.920 align:middle line:84% immediately, at the beginning of a talk. 00:02:18.920 --> 00:02:21.340 align:middle line:84% But there are a lot of talks that are sort 00:02:21.340 --> 00:02:23.200 align:middle line:90% of like, what is accessibility? 00:02:23.200 --> 00:02:24.628 align:middle line:90% And who has it? 00:02:24.628 --> 00:02:25.420 align:middle line:90% This kind of stuff. 00:02:25.420 --> 00:02:26.860 align:middle line:90% That's not this talk at all. 00:02:26.860 --> 00:02:28.090 align:middle line:90% This is a technical talk. 00:02:28.090 --> 00:02:29.620 align:middle line:84% This is a technical talk on how we 00:02:29.620 --> 00:02:35.350 align:middle line:84% make things accessible in nitty, gritty technical detail. 00:02:35.350 --> 00:02:40.390 align:middle line:84% So fair warning, I have tried to translate some of this stuff 00:02:40.390 --> 00:02:41.890 align:middle line:90% into English. 00:02:41.890 --> 00:02:45.280 align:middle line:84% But accessibility stuff, under the hood, 00:02:45.280 --> 00:02:47.020 align:middle line:90% is actually kind of complicated. 00:02:47.020 --> 00:02:48.880 align:middle line:84% It just is the nature of the beast. 00:02:48.880 --> 00:02:50.650 align:middle line:90% It just is how it is. 00:02:50.650 --> 00:02:55.660 align:middle line:84% So this talk is about what data science is and it's challenges 00:02:55.660 --> 00:02:59.560 align:middle line:84% and how accessibility works and then how we evaluate it 00:02:59.560 --> 00:03:04.337 align:middle line:84% in the context of graphics or anything data science related. 00:03:04.337 --> 00:03:05.920 align:middle line:84% Now, the reason I'm talking about some 00:03:05.920 --> 00:03:08.680 align:middle line:84% of the technical stuff is because even if you're just 00:03:08.680 --> 00:03:11.380 align:middle line:84% evaluating your team's accessibility of a product 00:03:11.380 --> 00:03:13.570 align:middle line:84% that you're making, believe it or not, 00:03:13.570 --> 00:03:15.910 align:middle line:84% a lot of the words and symbols and phrases 00:03:15.910 --> 00:03:18.910 align:middle line:84% that exist in code are actually the things 00:03:18.910 --> 00:03:21.240 align:middle line:84% that the accessibility tools that track things, 00:03:21.240 --> 00:03:22.990 align:middle line:90% they use the same terminology. 00:03:22.990 --> 00:03:24.430 align:middle line:90% And it's wacky. 00:03:24.430 --> 00:03:26.830 align:middle line:84% So I want to make that correspondence 00:03:26.830 --> 00:03:29.140 align:middle line:84% between those things clear so that when 00:03:29.140 --> 00:03:30.700 align:middle line:84% you do, even an accessibility check, 00:03:30.700 --> 00:03:32.440 align:middle line:84% you know what you're looking at and what 00:03:32.440 --> 00:03:36.620 align:middle line:84% you're looking for when you're doing this kind of work. 00:03:36.620 --> 00:03:38.292 align:middle line:90% So funny thing. 00:03:38.292 --> 00:03:40.000 align:middle line:84% So data science, if you're not familiar-- 00:03:40.000 --> 00:03:41.333 align:middle line:90% if you're not a data scientist-- 00:03:41.333 --> 00:03:42.510 align:middle line:90% I know some of you are. 00:03:42.510 --> 00:03:43.685 align:middle line:90% I know Gary is. 00:03:43.685 --> 00:03:46.060 align:middle line:84% And I know some of you have written some really cool data 00:03:46.060 --> 00:03:46.660 align:middle line:90% science tools. 00:03:46.660 --> 00:03:49.168 align:middle line:84% I know Neil Brown has these really cool spreadsheet 00:03:49.168 --> 00:03:50.710 align:middle line:84% type tools and stuff like that that I 00:03:50.710 --> 00:03:53.350 align:middle line:90% think are super impressive. 00:03:53.350 --> 00:03:55.530 align:middle line:84% But data science is a weird thing. 00:03:55.530 --> 00:03:57.550 align:middle line:84% One of the things we often say in publications, 00:03:57.550 --> 00:03:58.925 align:middle line:84% or what we've been saying lately, 00:03:58.925 --> 00:04:00.760 align:middle line:84% is it's kind of the wild, wild west. 00:04:00.760 --> 00:04:04.140 align:middle line:84% And the metaphor isn't that these people are all gun-totin', 00:04:04.140 --> 00:04:05.860 align:middle line:90% and I wouldn't really know that. 00:04:05.860 --> 00:04:09.910 align:middle line:84% But the metaphor is that it's not the most organized space 00:04:09.910 --> 00:04:11.710 align:middle line:90% of thinking on planet Earth. 00:04:11.710 --> 00:04:14.760 align:middle line:84% A lot of the libraries just use weird naming conventions, 00:04:14.760 --> 00:04:17.320 align:middle line:84% they're not consistent anywhere, they don't match textbooks. 00:04:17.320 --> 00:04:22.510 align:middle line:84% It's just sort of a strange discipline, broadly. 00:04:22.510 --> 00:04:25.360 align:middle line:84% But the problem is, though, the fact 00:04:25.360 --> 00:04:28.180 align:middle line:84% that it's the wild, wild west is kind of irrelevant because it's 00:04:28.180 --> 00:04:30.520 align:middle line:90% also really important. 00:04:30.520 --> 00:04:32.560 align:middle line:84% Empirical data and statistics are used pretty 00:04:32.560 --> 00:04:34.270 align:middle line:90% much in every discipline now. 00:04:34.270 --> 00:04:37.130 align:middle line:84% It's not really controversial to use evidence anymore, 00:04:37.130 --> 00:04:40.570 align:middle line:84% at least not since the Dark Ages, I guess. 00:04:40.570 --> 00:04:43.710 align:middle line:84% Funding agencies nowadays often require evidence. 00:04:43.710 --> 00:04:44.585 align:middle line:90% This is not shocking. 00:04:44.585 --> 00:04:47.140 align:middle line:84% We're in more a scientific community 00:04:47.140 --> 00:04:49.600 align:middle line:90% than we have been historically. 00:04:49.600 --> 00:04:51.670 align:middle line:84% And data science pays pretty well. 00:04:51.670 --> 00:04:55.000 align:middle line:84% So people want to do this kind of work because it's lucrative. 00:04:55.000 --> 00:04:57.580 align:middle line:84% You can make money being a data scientist. 00:04:57.580 --> 00:05:01.630 align:middle line:84% And then it's become so common that even in K-12, 00:05:01.630 --> 00:05:04.780 align:middle line:84% we're observing curriculum that may not be called data science. 00:05:04.780 --> 00:05:08.110 align:middle line:84% For example, code.org's unit 9, they 00:05:08.110 --> 00:05:09.850 align:middle line:84% don't call it a data science unit, 00:05:09.850 --> 00:05:12.040 align:middle line:84% but there's clearly a relationship 00:05:12.040 --> 00:05:14.360 align:middle line:84% between data science and the kinds of content 00:05:14.360 --> 00:05:15.360 align:middle line:90% that students are doing. 00:05:15.360 --> 00:05:19.720 align:middle line:84% It's sort of an early intro to data science in a sense, right? 00:05:19.720 --> 00:05:22.340 align:middle line:90% Depending on how you look at it. 00:05:22.340 --> 00:05:25.250 align:middle line:84% So then the question becomes then, well, what is it exactly? 00:05:25.250 --> 00:05:26.810 align:middle line:90% What is data science? 00:05:26.810 --> 00:05:29.900 align:middle line:84% And, you know, there is no universal definition, 00:05:29.900 --> 00:05:30.860 align:middle line:90% I would say. 00:05:30.860 --> 00:05:33.620 align:middle line:84% But there are groups that are kind of 00:05:33.620 --> 00:05:36.400 align:middle line:84% trying to define this in some formal sense or the other. 00:05:36.400 --> 00:05:39.380 align:middle line:84% And I'll mention, too, there's a new report out of Harvard where 00:05:39.380 --> 00:05:42.190 align:middle line:84% they tried to kind of talk about this. 00:05:42.190 --> 00:05:44.360 align:middle line:84% And the one that I'm probably more familiar with, 00:05:44.360 --> 00:05:48.170 align:middle line:84% just because I'm sort of loosely connected to that committee, 00:05:48.170 --> 00:05:50.750 align:middle line:84% is the ACM Data Science Task Force. 00:05:50.750 --> 00:05:54.620 align:middle line:84% And they define it as having 11 knowledge areas. 00:05:54.620 --> 00:05:57.230 align:middle line:84% And of course, keep in mind, this 00:05:57.230 --> 00:05:58.790 align:middle line:90% is a bit subjective, right? 00:05:58.790 --> 00:06:00.680 align:middle line:84% Exactly whether these are the right 11 00:06:00.680 --> 00:06:02.780 align:middle line:90% is unclear, but not the point. 00:06:02.780 --> 00:06:06.830 align:middle line:84% They include things like data analysis, presentation, or AI, 00:06:06.830 --> 00:06:11.030 align:middle line:84% or big data, or computing fundamentals, acquisition, 00:06:11.030 --> 00:06:13.880 align:middle line:84% management, and governance, data mining, data privacy, 00:06:13.880 --> 00:06:16.530 align:middle line:84% machine learning, professionalism, and stuff 00:06:16.530 --> 00:06:17.030 align:middle line:90% like that. 00:06:17.030 --> 00:06:19.370 align:middle line:84% If you're not familiar with what some of the thinking 00:06:19.370 --> 00:06:22.610 align:middle line:84% is, at least in academia, on what data science is, 00:06:22.610 --> 00:06:26.600 align:middle line:84% you can actually read the draft report of the ACM Data Science 00:06:26.600 --> 00:06:29.980 align:middle line:90% Task Force at dstf.acm.org. 00:06:29.980 --> 00:06:31.540 align:middle line:90% That's draft two. 00:06:31.540 --> 00:06:33.920 align:middle line:84% I think draft three or four will end up being final, 00:06:33.920 --> 00:06:40.405 align:middle line:84% although I'm not positive on the of the exact stuff there. 00:06:40.405 --> 00:06:42.030 align:middle line:84% So what do people do with data science? 00:06:42.030 --> 00:06:45.870 align:middle line:84% Well, this is also difficult to answer. 00:06:45.870 --> 00:06:47.940 align:middle line:84% There are some interesting surveys on it, 00:06:47.940 --> 00:06:50.668 align:middle line:84% and there's some really good reports, too, 00:06:50.668 --> 00:06:52.710 align:middle line:84% that have been coming out over the last few years 00:06:52.710 --> 00:06:54.900 align:middle line:84% on the kinds of things that data scientists do. 00:06:54.900 --> 00:06:58.530 align:middle line:84% But one example of this might be the Stack Overflow survey 00:06:58.530 --> 00:07:00.360 align:middle line:84% from a few years ago where they looked 00:07:00.360 --> 00:07:02.520 align:middle line:84% at just what percent of developers 00:07:02.520 --> 00:07:03.860 align:middle line:90% do stuff like statistics. 00:07:03.860 --> 00:07:07.590 align:middle line:84% And it turns out to be around 11%-ish, 11.3, 00:07:07.590 --> 00:07:09.280 align:middle line:90% according to their survey. 00:07:09.280 --> 00:07:13.800 align:middle line:84% I read this is Stack Overflow, so you can decide for yourself 00:07:13.800 --> 00:07:16.590 align:middle line:84% whether that sample is actually generalizable. 00:07:16.590 --> 00:07:19.680 align:middle line:84% But nonetheless, it's one data point that we can think about. 00:07:19.680 --> 00:07:23.100 align:middle line:84% And of that, I think it's ballpark nearly half ended up 00:07:23.100 --> 00:07:26.910 align:middle line:90% using Python, R, or Matlab. 00:07:26.910 --> 00:07:31.440 align:middle line:84% And there's surprisingly little evidence on the sort 00:07:31.440 --> 00:07:34.290 align:middle line:84% of tools on language designs, on the accessibility, 00:07:34.290 --> 00:07:35.970 align:middle line:84% really, on almost anything in relation 00:07:35.970 --> 00:07:38.040 align:middle line:84% to how people are doing data science. 00:07:38.040 --> 00:07:39.715 align:middle line:84% It's only been in the last few years 00:07:39.715 --> 00:07:42.090 align:middle line:84% that you've been really seeing a lot of publications come 00:07:42.090 --> 00:07:44.640 align:middle line:84% out that contain data on the kinds of things 00:07:44.640 --> 00:07:45.660 align:middle line:90% that data scientists do. 00:07:45.660 --> 00:07:46.577 align:middle line:90% There's a few of them. 00:07:46.577 --> 00:07:48.630 align:middle line:84% We've written some of them, but we're not alone. 00:07:48.630 --> 00:07:50.755 align:middle line:84% Austin Henley has a really good paper at out of CHI 00:07:50.755 --> 00:07:54.810 align:middle line:84% And some other groups, as well, have done some cool work here. 00:07:54.810 --> 00:07:58.610 align:middle line:84% So one of the things that we did after talking 00:07:58.610 --> 00:08:02.210 align:middle line:84% to Gary Ritchie and his team at our studio over the summer 00:08:02.210 --> 00:08:05.690 align:middle line:84% was we did a quick analysis of all the tools that we find. 00:08:05.690 --> 00:08:08.960 align:middle line:84% And we went in and just looked at what is accessible 00:08:08.960 --> 00:08:10.280 align:middle line:90% and what is not across these. 00:08:10.280 --> 00:08:13.370 align:middle line:84% In part, just because I was curious, but also because 00:08:13.370 --> 00:08:15.560 align:middle line:84% it's not really that difficult to download them. 00:08:15.560 --> 00:08:17.960 align:middle line:84% And say, does it turn on with a screen reader 00:08:17.960 --> 00:08:19.550 align:middle line:84% and what other problems does it have? 00:08:19.550 --> 00:08:20.870 align:middle line:90% Things like that. 00:08:20.870 --> 00:08:23.240 align:middle line:84% And so I'm not going to read this whole list. 00:08:23.240 --> 00:08:25.700 align:middle line:84% You can reference to the slides if you want to. 00:08:25.700 --> 00:08:27.410 align:middle line:84% But I want to give you a sample of some 00:08:27.410 --> 00:08:31.010 align:middle line:84% of these verbally, just in case there's viewers 00:08:31.010 --> 00:08:32.520 align:middle line:90% that can't see the screen. 00:08:32.520 --> 00:08:35.659 align:middle line:84% These are tools like RStudio, CODAP, iNZight, DataDesk, 00:08:35.659 --> 00:08:38.419 align:middle line:84% Fathom, Jupyter Notebooks, IntelliJ, SpyderJ, Thony, Atom, 00:08:38.419 --> 00:08:42.473 align:middle line:84% JMP, MATLAP, RapidMiner, SAS, SPSS, Statview, TinkerPlots. 00:08:42.473 --> 00:08:43.640 align:middle line:90% And there's others, as well. 00:08:43.640 --> 00:08:45.950 align:middle line:84% There's groups from Tableau, there's 00:08:45.950 --> 00:08:49.010 align:middle line:84% groups out of SAS that do what's called the graphics 00:08:49.010 --> 00:08:52.610 align:middle line:84% accelerator, which is a really cool tool related to especially 00:08:52.610 --> 00:08:54.980 align:middle line:84% sonification, but also some speech, as well, 00:08:54.980 --> 00:08:58.562 align:middle line:90% in terms of sonifying data. 00:08:58.562 --> 00:09:00.020 align:middle line:84% There's also whole communities that 00:09:00.020 --> 00:09:01.562 align:middle line:84% have been thinking about sonification 00:09:01.562 --> 00:09:03.540 align:middle line:84% for a long time with the ICAD community, 00:09:03.540 --> 00:09:06.170 align:middle line:84% the International Conference on Auditory Display 00:09:06.170 --> 00:09:07.070 align:middle line:90% and stuff like that. 00:09:07.070 --> 00:09:09.470 align:middle line:84% I'm not going to go through all that research literature. 00:09:09.470 --> 00:09:11.090 align:middle line:90% There is some of it out there. 00:09:11.090 --> 00:09:14.480 align:middle line:84% But I just want to make it clear that there is a list of these, 00:09:14.480 --> 00:09:16.370 align:middle line:84% and when you look at the accessibility 00:09:16.370 --> 00:09:19.820 align:middle line:84% across these tools, there is a lot 00:09:19.820 --> 00:09:21.830 align:middle line:90% of accessibility challenges. 00:09:21.830 --> 00:09:23.360 align:middle line:84% And when I say challenges, I don't 00:09:23.360 --> 00:09:25.520 align:middle line:84% mean people are doing a good or bad job. 00:09:25.520 --> 00:09:26.810 align:middle line:90% That's not really my point. 00:09:26.810 --> 00:09:29.540 align:middle line:84% My point is that in some cases, groups just 00:09:29.540 --> 00:09:31.620 align:middle line:84% didn't have support for certain things. 00:09:31.620 --> 00:09:33.590 align:middle line:84% And in other cases, it's just really hard 00:09:33.590 --> 00:09:36.170 align:middle line:84% to figure out what it should be to be accessible. 00:09:36.170 --> 00:09:38.480 align:middle line:84% So there's a lot of accessibility challenges 00:09:38.480 --> 00:09:39.860 align:middle line:90% in data science. 00:09:39.860 --> 00:09:41.940 align:middle line:90% But don't take my word for it. 00:09:41.940 --> 00:09:44.585 align:middle line:84% There's this wonderful fellow named Jonathan Godfrey, 00:09:44.585 --> 00:09:45.830 align:middle line:90% who I don't really know. 00:09:45.830 --> 00:09:48.380 align:middle line:84% I've emailed back and forth with him a time or two. 00:09:48.380 --> 00:09:50.570 align:middle line:84% And he's written a few papers on this topic. 00:09:50.570 --> 00:09:51.770 align:middle line:90% And they're really neat. 00:09:51.770 --> 00:09:54.800 align:middle line:84% If you don't know them, they're worth taking a look at. 00:09:54.800 --> 00:09:57.080 align:middle line:84% One of them is this paper that he 00:09:57.080 --> 00:10:00.980 align:middle line:84% wrote in 2013 called Statistical Software from a Blind Person's 00:10:00.980 --> 00:10:02.030 align:middle line:90% Perspective. 00:10:02.030 --> 00:10:03.530 align:middle line:84% In that particular case, I think he 00:10:03.530 --> 00:10:06.112 align:middle line:84% was using R, one of the variations, 00:10:06.112 --> 00:10:07.820 align:middle line:84% I don't think it was RStudio at the time. 00:10:07.820 --> 00:10:09.890 align:middle line:84% I think it was one of the other tools. 00:10:09.890 --> 00:10:11.280 align:middle line:90% But that's not really the point. 00:10:11.280 --> 00:10:15.080 align:middle line:84% The point was that he was using one of the one of the R 00:10:15.080 --> 00:10:16.940 align:middle line:90% just default packages. 00:10:16.940 --> 00:10:19.910 align:middle line:84% And Jonathan has a bunch of newer work, as well, where 00:10:19.910 --> 00:10:22.160 align:middle line:84% he talks about how do you do accessible graphing 00:10:22.160 --> 00:10:24.105 align:middle line:84% and how do you do accessible this and that. 00:10:24.105 --> 00:10:25.730 align:middle line:84% And they're really fascinating and give 00:10:25.730 --> 00:10:28.400 align:middle line:84% a lot of really good ideas about what 00:10:28.400 --> 00:10:32.060 align:middle line:84% it might mean for data science to become more accessible. 00:10:32.060 --> 00:10:36.170 align:middle line:84% And they really, at their core, I think, at their core-- 00:10:36.170 --> 00:10:39.440 align:middle line:84% and, you know, he might totally disagree with me. 00:10:39.440 --> 00:10:41.000 align:middle line:84% At their core, they really come down 00:10:41.000 --> 00:10:46.460 align:middle line:84% to having more structure inside of these kind of tools. 00:10:46.460 --> 00:10:48.600 align:middle line:90% And I can get to that in a bit. 00:10:48.600 --> 00:10:50.090 align:middle line:84% The key lesson, I think, is really 00:10:50.090 --> 00:10:52.340 align:middle line:84% that there's still pretty major accessibility barriers 00:10:52.340 --> 00:10:53.180 align:middle line:90% in data science. 00:10:53.180 --> 00:10:55.860 align:middle line:90% Just massive, massive barriers. 00:10:55.860 --> 00:10:59.167 align:middle line:84% So the next thing I want to do is I actually have a pop quiz. 00:10:59.167 --> 00:11:00.500 align:middle line:90% And I know you'll all love that. 00:11:00.500 --> 00:11:02.640 align:middle line:84% That's exactly what you want in a webinar. 00:11:02.640 --> 00:11:04.970 align:middle line:84% So if you don't know how to do it, 00:11:04.970 --> 00:11:07.250 align:middle line:84% go ahead and pull up in the screen. 00:11:07.250 --> 00:11:08.762 align:middle line:90% There's a chat window. 00:11:08.762 --> 00:11:10.220 align:middle line:84% If you're a screen reader user, you 00:11:10.220 --> 00:11:12.763 align:middle line:84% press Alt h to get to the chat window. 00:11:12.763 --> 00:11:14.930 align:middle line:84% And if you're not, you can either press that hotkey, 00:11:14.930 --> 00:11:16.353 align:middle line:90% or you can just go. 00:11:16.353 --> 00:11:18.770 align:middle line:84% And what I'm going to do is I'm going to have three sides. 00:11:18.770 --> 00:11:21.062 align:middle line:84% And what I'm going to do is I'm going to give everybody 00:11:21.062 --> 00:11:23.810 align:middle line:84% about 30 seconds to actually discuss 00:11:23.810 --> 00:11:28.260 align:middle line:84% what they imagine the accessibility barriers are 00:11:28.260 --> 00:11:32.970 align:middle line:90% to a particular thing. 00:11:32.970 --> 00:11:35.720 align:middle line:84% And so in the first slide I have on the screen, 00:11:35.720 --> 00:11:38.310 align:middle line:84% I'm going to read the text in case you can't see it, 00:11:38.310 --> 00:11:39.018 align:middle line:90% some code. 00:11:39.018 --> 00:11:40.560 align:middle line:84% And I want the accessibility barriers 00:11:40.560 --> 00:11:44.190 align:middle line:84% to be discussed for just the code, not the tools around it, 00:11:44.190 --> 00:11:46.470 align:middle line:84% not how you interact with it or anything like that. 00:11:46.470 --> 00:11:49.080 align:middle line:84% Just the words and symbols that are on the screen. 00:11:49.080 --> 00:11:51.670 align:middle line:84% Are there accessibility barriers here? 00:11:51.670 --> 00:11:56.640 align:middle line:84% So the code is library(Lock5Data), 00:11:56.640 --> 00:12:04.380 align:middle line:84% data("SandwichAnts") m1 less than -aov(Ants~Fillin 00:12:04.380 --> 00:12:11.080 align:middle line:84% g,data=SandwichAnts), summary(m1). 00:12:11.080 --> 00:12:14.600 align:middle line:90% 00:12:14.600 --> 00:12:15.290 align:middle line:90% OK? 00:12:15.290 --> 00:12:20.330 align:middle line:84% Now you all have 30 seconds to come up with any ideas 00:12:20.330 --> 00:12:23.990 align:middle line:84% that you think might be related to accessibility and toss them 00:12:23.990 --> 00:12:24.590 align:middle line:90% in the chat. 00:12:24.590 --> 00:12:26.840 align:middle line:90% I'm going to be reading that. 00:12:26.840 --> 00:12:27.590 align:middle line:90% So give it a shot. 00:12:27.590 --> 00:12:29.090 align:middle line:84% I know it's a little weird, but it's 00:12:29.090 --> 00:12:32.530 align:middle line:90% a good way to get interaction. 00:12:32.530 --> 00:12:34.530 align:middle line:84% Screen readers can't read that in paragraph. 00:12:34.530 --> 00:12:36.070 align:middle line:90% Aov is not pronounceable. 00:12:36.070 --> 00:12:38.680 align:middle line:84% Leaving spaces versus leaving tabs. 00:12:38.680 --> 00:12:40.840 align:middle line:84% Numerals versus spelling numbers out. 00:12:40.840 --> 00:12:43.270 align:middle line:90% How do you pronounce tilde? 00:12:43.270 --> 00:12:45.210 align:middle line:90% That's good. 00:12:45.210 --> 00:12:46.095 align:middle line:90% You got 10 seconds. 00:12:46.095 --> 00:12:49.320 align:middle line:90% 00:12:49.320 --> 00:12:50.880 align:middle line:90% Summarym1 has no spaces. 00:12:50.880 --> 00:12:54.920 align:middle line:90% Tilde's not used often. 00:12:54.920 --> 00:12:57.840 align:middle line:90% OK, well we have to stop. 00:12:57.840 --> 00:12:59.940 align:middle line:84% If I'm new to the field, what does this even mean? 00:12:59.940 --> 00:13:00.740 align:middle line:90% Aov? 00:13:00.740 --> 00:13:03.400 align:middle line:90% Yeah, exactly. 00:13:03.400 --> 00:13:04.140 align:middle line:90% Oh, that's Jenna. 00:13:04.140 --> 00:13:05.390 align:middle line:90% Hi, Jenna. 00:13:05.390 --> 00:13:05.890 align:middle line:90% OK. 00:13:05.890 --> 00:13:07.350 align:middle line:90% So let's break that down. 00:13:07.350 --> 00:13:08.860 align:middle line:84% Now there is actually quite a bit 00:13:08.860 --> 00:13:12.285 align:middle line:84% of evidence on words and symbols in the academic literature 00:13:12.285 --> 00:13:13.660 align:middle line:84% related to programming languages. 00:13:13.660 --> 00:13:16.750 align:middle line:84% And I'll just briefly mention what some of these are. 00:13:16.750 --> 00:13:21.280 align:middle line:84% Oftentimes, people might assume that a particular word has 00:13:21.280 --> 00:13:25.390 align:middle line:84% a particular meaning to humans, but the problem is humans vary. 00:13:25.390 --> 00:13:27.500 align:middle line:90% So I'll give you an example. 00:13:27.500 --> 00:13:31.540 align:middle line:84% Less than dash might look visually like an arrow, 00:13:31.540 --> 00:13:34.270 align:middle line:84% but actually, even to novices, the less than dash 00:13:34.270 --> 00:13:35.963 align:middle line:90% does not mean assigned to. 00:13:35.963 --> 00:13:36.880 align:middle line:90% It does not mean that. 00:13:36.880 --> 00:13:39.490 align:middle line:84% To novices, it's been formally tested in studies, right? 00:13:39.490 --> 00:13:41.560 align:middle line:90% We did one of them. 00:13:41.560 --> 00:13:43.762 align:middle line:84% And in addition, a lot of the words that 00:13:43.762 --> 00:13:45.220 align:middle line:84% are used in programming languages-- 00:13:45.220 --> 00:13:47.020 align:middle line:84% this happens to be R, but it's relevant. 00:13:47.020 --> 00:13:50.020 align:middle line:84% Matlab and Python are very similar in terms 00:13:50.020 --> 00:13:52.120 align:middle line:90% of their choices. 00:13:52.120 --> 00:13:55.120 align:middle line:84% Terms like a aov are shortened acronyms 00:13:55.120 --> 00:13:57.400 align:middle line:84% that don't match either the academic literature, 00:13:57.400 --> 00:14:00.490 align:middle line:84% don't match textbooks, and don't match Matlab and Python, 00:14:00.490 --> 00:14:01.580 align:middle line:90% either. 00:14:01.580 --> 00:14:04.930 align:middle line:84% And so these terms actually don't mean anything to people. 00:14:04.930 --> 00:14:07.840 align:middle line:84% When you have certain kinds of characters like tilde, 00:14:07.840 --> 00:14:10.480 align:middle line:84% tilde has a particular semantic in R 00:14:10.480 --> 00:14:12.610 align:middle line:84% and its meaning is kind of wacky. 00:14:12.610 --> 00:14:17.290 align:middle line:84% And I have to admit, I only mostly understand it, I think. 00:14:17.290 --> 00:14:20.140 align:middle line:84% I know how to use it and when, but exactly what 00:14:20.140 --> 00:14:22.390 align:middle line:84% the deeper language meaning is is maybe 00:14:22.390 --> 00:14:23.570 align:middle line:90% a little bit elusive to me. 00:14:23.570 --> 00:14:24.383 align:middle line:90% I don't know. 00:14:24.383 --> 00:14:26.050 align:middle line:84% But the point is, to a novice, the tilde 00:14:26.050 --> 00:14:28.160 align:middle line:84% doesn't mean what you probably think it means, 00:14:28.160 --> 00:14:31.090 align:middle line:90% I would imagine. 00:14:31.090 --> 00:14:31.900 align:middle line:90% Stuff like that. 00:14:31.900 --> 00:14:35.410 align:middle line:84% Oh, there's also Good Data by Neil Brown and Brett Becker 00:14:35.410 --> 00:14:39.190 align:middle line:84% and some of those kind of folks on parentheses matching being 00:14:39.190 --> 00:14:40.810 align:middle line:90% a particular issue for novices. 00:14:40.810 --> 00:14:43.300 align:middle line:84% This isn't just accessibility, but it's for everybody. 00:14:43.300 --> 00:14:46.750 align:middle line:84% But it is an issue that shows up in their data quite a bit. 00:14:46.750 --> 00:14:48.250 align:middle line:84% So those are a couple of things just 00:14:48.250 --> 00:14:51.430 align:middle line:84% related to the text, just related to accessibility. 00:14:51.430 --> 00:14:53.440 align:middle line:84% It's not an indictment of any particular group. 00:14:53.440 --> 00:14:55.600 align:middle line:84% It's just observations based on facts 00:14:55.600 --> 00:14:58.840 align:middle line:84% from the literature about the kinds of issues 00:14:58.840 --> 00:15:01.280 align:middle line:90% that people have. 00:15:01.280 --> 00:15:02.570 align:middle line:90% So let's go to the next one. 00:15:02.570 --> 00:15:03.570 align:middle line:90% So that's program input. 00:15:03.570 --> 00:15:04.987 align:middle line:84% So now let's go to program output. 00:15:04.987 --> 00:15:06.590 align:middle line:84% And for this one, I actually have 00:15:06.590 --> 00:15:09.470 align:middle line:84% to change what I'm sharing on the screen for a second 00:15:09.470 --> 00:15:13.560 align:middle line:84% because I want you to observe it in a different kind of way. 00:15:13.560 --> 00:15:17.095 align:middle line:84% So I'm going to stop sharing and I'm going to pull up this code. 00:15:17.095 --> 00:15:18.470 align:middle line:84% And what I'm going to be doing is 00:15:18.470 --> 00:15:21.170 align:middle line:84% I'm going to actually pull this up in a screen reader. 00:15:21.170 --> 00:15:24.890 align:middle line:84% And I'm going to share a screen as a whole 00:15:24.890 --> 00:15:29.150 align:middle line:84% but I'm going to do it with shared sound. 00:15:29.150 --> 00:15:34.570 align:middle line:84% So I'm going to pull up Notepad and I'm going to turn on NVDA. 00:15:34.570 --> 00:15:36.337 align:middle line:90% NVDA: [INAUDIBLE] 00:15:36.337 --> 00:15:37.920 align:middle line:84% ANDREAS STEFIK: And I've intentionally 00:15:37.920 --> 00:15:41.610 align:middle line:84% made NVDA ridiculously slow to try 00:15:41.610 --> 00:15:44.730 align:middle line:84% to make it easier to understand if you're not a screen reader 00:15:44.730 --> 00:15:45.338 align:middle line:90% user. 00:15:45.338 --> 00:15:46.755 align:middle line:84% So I'm going to dump in some code. 00:15:46.755 --> 00:15:48.370 align:middle line:90% I'm going to dump it to the top. 00:15:48.370 --> 00:15:50.110 align:middle line:90% NVDA: [INAUDIBLE] 00:15:50.110 --> 00:15:51.948 align:middle line:90% 00:15:51.948 --> 00:15:52.740 align:middle line:90% ANDREAS STEFIK: OK? 00:15:52.740 --> 00:15:54.992 align:middle line:84% Now the first thing is if I'm navigating with a screen 00:15:54.992 --> 00:15:57.450 align:middle line:84% reader, I'm not going to give you a screen reader tutorial. 00:15:57.450 --> 00:15:59.283 align:middle line:84% But I'm going to kind of navigate through it 00:15:59.283 --> 00:16:01.110 align:middle line:84% a little bit to give you an idea of how 00:16:01.110 --> 00:16:02.490 align:middle line:90% you would interact with this. 00:16:02.490 --> 00:16:04.198 align:middle line:84% And then I'm going to give you another 30 00:16:04.198 --> 00:16:07.698 align:middle line:84% seconds to try to derive for yourself mentally how you would 00:16:07.698 --> 00:16:09.240 align:middle line:84% do this with a screen reader and what 00:16:09.240 --> 00:16:11.130 align:middle line:90% the accessibility problems are. 00:16:11.130 --> 00:16:15.240 align:middle line:84% So if I want to know, for example, what the value of f 00:16:15.240 --> 00:16:18.990 align:middle line:84% is in this ANOVA that I've put in, what I have to do 00:16:18.990 --> 00:16:20.580 align:middle line:84% is I can either go press up and down. 00:16:20.580 --> 00:16:22.890 align:middle line:90% NVDA: [INAUDIBLE] 00:16:22.890 --> 00:16:24.270 align:middle line:90% 00:16:24.270 --> 00:16:26.520 align:middle line:84% ANDREAS STEFIK: Mentally track the columns 00:16:26.520 --> 00:16:27.450 align:middle line:90% and then press down. 00:16:27.450 --> 00:16:29.090 align:middle line:90% NVDA: [INAUDIBLE] 00:16:29.090 --> 00:16:33.018 align:middle line:90% 00:16:33.018 --> 00:16:33.810 align:middle line:90% ANDREAS STEFIK: OK? 00:16:33.810 --> 00:16:35.970 align:middle line:84% Or I can go in and I can navigate 00:16:35.970 --> 00:16:37.110 align:middle line:90% by things like columns. 00:16:37.110 --> 00:16:40.020 align:middle line:90% NVDA: [INAUDIBLE] 00:16:40.020 --> 00:16:41.700 align:middle line:90% 00:16:41.700 --> 00:16:45.580 align:middle line:84% ANDREAS STEFIK: If I do this, I then have to mentally line up 00:16:45.580 --> 00:16:49.140 align:middle line:84% that mean SQ and sum SQ are actually columns, 00:16:49.140 --> 00:16:52.020 align:middle line:84% but they're two screen reader calls because of the press Ctrl 00:16:52.020 --> 00:16:53.350 align:middle line:90% right twice. 00:16:53.350 --> 00:16:54.900 align:middle line:90% And then I have to go down-- 00:16:54.900 --> 00:16:56.615 align:middle line:90% NVDA: [INAUDIBLE] 00:16:56.615 --> 00:16:57.990 align:middle line:84% ANDREAS STEFIK: --go to the left, 00:16:57.990 --> 00:16:59.790 align:middle line:84% and then I have to line up those columns 00:16:59.790 --> 00:17:03.450 align:middle line:84% with the imagined columns by lining them up a different way 00:17:03.450 --> 00:17:04.513 align:middle line:90% mentally at the bottom. 00:17:04.513 --> 00:17:06.790 align:middle line:90% NVDA: [INAUDIBLE] 1561. 00:17:06.790 --> 00:17:08.349 align:middle line:90% 785.627. 00:17:08.349 --> 00:17:10.349 align:middle line:84% ANDREAS STEFIK: And if I do that mental line up, 00:17:10.349 --> 00:17:13.650 align:middle line:84% I can track in my head which columns match to which. 00:17:13.650 --> 00:17:15.839 align:middle line:84% And then all of the sudden, I have 00:17:15.839 --> 00:17:18.359 align:middle line:84% text from R translated into something 00:17:18.359 --> 00:17:20.550 align:middle line:84% accessible through a screen reader. 00:17:20.550 --> 00:17:21.050 align:middle line:90% OK. 00:17:21.050 --> 00:17:22.520 align:middle line:84% That's a very, very brief tutorial 00:17:22.520 --> 00:17:23.260 align:middle line:84% about how you might get some output-- 00:17:23.260 --> 00:17:24.109 align:middle line:90% NVDA: [INAUDIBLE] 00:17:24.109 --> 00:17:28.369 align:middle line:84% ANDREAS STEFIK: --from something like R. So 30 seconds again. 00:17:28.369 --> 00:17:30.500 align:middle line:84% Go ahead and chat any other issues 00:17:30.500 --> 00:17:33.050 align:middle line:84% you think with having a screen reader with specifically 00:17:33.050 --> 00:17:35.030 align:middle line:84% the unstructured output that is often 00:17:35.030 --> 00:17:38.540 align:middle line:90% given from these environments. 00:17:38.540 --> 00:17:39.260 align:middle line:90% 30 seconds. 00:17:39.260 --> 00:17:43.253 align:middle line:90% 00:17:43.253 --> 00:17:45.170 align:middle line:84% Less than, dashes, and arrow but would be read 00:17:45.170 --> 00:17:46.340 align:middle line:90% as two separate characters. 00:17:46.340 --> 00:17:49.235 align:middle line:90% Yep, that's definitely true. 00:17:49.235 --> 00:17:51.360 align:middle line:84% That's input, but I guess that's from the last one. 00:17:51.360 --> 00:17:53.010 align:middle line:90% I see people typing, or some. 00:17:53.010 --> 00:17:57.170 align:middle line:90% 00:17:57.170 --> 00:17:59.960 align:middle line:84% Everything on the screen is too hard to read. 00:17:59.960 --> 00:18:00.460 align:middle line:90% Oh. 00:18:00.460 --> 00:18:01.668 align:middle line:90% I'll try to make that bigger. 00:18:01.668 --> 00:18:02.720 align:middle line:90% Sorry about that. 00:18:02.720 --> 00:18:04.580 align:middle line:84% If you aren't used to the field, arranging this in your head 00:18:04.580 --> 00:18:05.288 align:middle line:90% is not intuitive. 00:18:05.288 --> 00:18:06.350 align:middle line:90% Definitely. 00:18:06.350 --> 00:18:07.940 align:middle line:84% Some of the spaces within columns, 00:18:07.940 --> 00:18:10.210 align:middle line:84% space characters, some of those column separators tab. 00:18:10.210 --> 00:18:11.030 align:middle line:90% Oh, yeah. 00:18:11.030 --> 00:18:12.560 align:middle line:90% Actually, that's probably true. 00:18:12.560 --> 00:18:14.690 align:middle line:84% None of the visual layout clues are automatically 00:18:14.690 --> 00:18:15.730 align:middle line:84% available to screen reader users. 00:18:15.730 --> 00:18:16.230 align:middle line:90% Yep. 00:18:16.230 --> 00:18:18.260 align:middle line:84% High cognitive load, almost certainly. 00:18:18.260 --> 00:18:22.330 align:middle line:84% Mapping of rows and columns is difficult. Definitely true. 00:18:22.330 --> 00:18:23.560 align:middle line:90% It's not on a table. 00:18:23.560 --> 00:18:24.220 align:middle line:90% Yeah. 00:18:24.220 --> 00:18:27.280 align:middle line:84% That's really the simplest answer to this, actually. 00:18:27.280 --> 00:18:28.870 align:middle line:84% Difficult to go down within a column. 00:18:28.870 --> 00:18:30.340 align:middle line:90% 100% true. 00:18:30.340 --> 00:18:32.440 align:middle line:84% Empty cells cause off by one errors. 00:18:32.440 --> 00:18:33.050 align:middle line:90% Totally. 00:18:33.050 --> 00:18:33.550 align:middle line:90% Yeah. 00:18:33.550 --> 00:18:35.170 align:middle line:84% All these things sound right to me. 00:18:35.170 --> 00:18:37.390 align:middle line:90% So why don't we stop there. 00:18:37.390 --> 00:18:40.460 align:middle line:84% OK now there's one more of these exercises that I want to do. 00:18:40.460 --> 00:18:41.520 align:middle line:84% And I'm going to share my screen again. 00:18:41.520 --> 00:18:43.603 align:middle line:84% And I'll make it bigger so that everybody can see. 00:18:43.603 --> 00:18:46.140 align:middle line:90% 00:18:46.140 --> 00:18:48.000 align:middle line:90% And I'll-- oops, wait. 00:18:48.000 --> 00:18:49.470 align:middle line:90% I need to Present Mode. 00:18:49.470 --> 00:18:52.195 align:middle line:90% 00:18:52.195 --> 00:18:55.380 align:middle line:90% And-- sorry, just one second. 00:18:55.380 --> 00:18:58.980 align:middle line:90% 00:18:58.980 --> 00:18:59.480 align:middle line:90% OK. 00:18:59.480 --> 00:19:03.210 align:middle line:84% And when I do this, I have now on the screen one more, 00:19:03.210 --> 00:19:05.570 align:middle line:90% in this case, it's a graph. 00:19:05.570 --> 00:19:09.502 align:middle line:84% And this graph actually comes from code.org's data unit 9, 00:19:09.502 --> 00:19:10.710 align:middle line:90% although that doesn't matter. 00:19:10.710 --> 00:19:12.110 align:middle line:90% It's really just a graph. 00:19:12.110 --> 00:19:14.480 align:middle line:84% And I'll describe it in English so 00:19:14.480 --> 00:19:18.470 align:middle line:84% that people that can't see it can know what's there. 00:19:18.470 --> 00:19:20.450 align:middle line:84% Basically, this is a graph that looks 00:19:20.450 --> 00:19:22.220 align:middle line:84% kind of like a longitudinal graph where 00:19:22.220 --> 00:19:25.280 align:middle line:84% it's tracking certain blips in the data over time. 00:19:25.280 --> 00:19:28.580 align:middle line:90% It's got dates on the bottom. 00:19:28.580 --> 00:19:31.170 align:middle line:84% February 15th, 20, dot, dot, dot. 00:19:31.170 --> 00:19:32.670 align:middle line:84% And then on the right on the bottom, 00:19:32.670 --> 00:19:35.030 align:middle line:90% it's September 1st, 2019. 00:19:35.030 --> 00:19:39.200 align:middle line:84% On the left axis, there's 100, 75, 50, and 25. 00:19:39.200 --> 00:19:41.000 align:middle line:84% And then it's basically a line chart 00:19:41.000 --> 00:19:43.130 align:middle line:84% that has a bunch of little wobbling in it. 00:19:43.130 --> 00:19:44.930 align:middle line:84% And then it has these occasional blips. 00:19:44.930 --> 00:19:48.350 align:middle line:84% And there's five of them that seem to go at a relatively 00:19:48.350 --> 00:19:51.110 align:middle line:90% periodic pattern, OK? 00:19:51.110 --> 00:19:53.330 align:middle line:84% And there's a couple of things to notice here 00:19:53.330 --> 00:19:54.080 align:middle line:90% that I'll mention. 00:19:54.080 --> 00:19:55.970 align:middle line:84% But again, I want to give you 30 seconds 00:19:55.970 --> 00:19:58.790 align:middle line:84% to figure out exactly what are the challenges here 00:19:58.790 --> 00:20:02.330 align:middle line:84% with making this accessible and what makes this-- 00:20:02.330 --> 00:20:04.550 align:middle line:84% what are the challenges here with this kind of stuff? 00:20:04.550 --> 00:20:05.390 align:middle line:90% So 30 seconds. 00:20:05.390 --> 00:20:05.890 align:middle line:90% Go. 00:20:05.890 --> 00:20:14.195 align:middle line:90% 00:20:14.195 --> 00:20:16.570 align:middle line:84% If it's just an image, no better data for screen readers. 00:20:16.570 --> 00:20:17.040 align:middle line:90% Yep. 00:20:17.040 --> 00:20:19.582 align:middle line:84% Glance exposure is not something that can be effectively done 00:20:19.582 --> 00:20:20.940 align:middle line:90% by a text-voice descriptions. 00:20:20.940 --> 00:20:21.863 align:middle line:90% Yep. 00:20:21.863 --> 00:20:23.530 align:middle line:84% People that have never seen a line chart 00:20:23.530 --> 00:20:24.580 align:middle line:90% won't understand the structure. 00:20:24.580 --> 00:20:25.140 align:middle line:90% Yep. 00:20:25.140 --> 00:20:27.473 align:middle line:84% In describing it, you put your own interpretation on it. 00:20:27.473 --> 00:20:28.060 align:middle line:90% Yeah, exactly. 00:20:28.060 --> 00:20:31.610 align:middle line:84% Even if you have a descript-- yeah. 00:20:31.610 --> 00:20:33.420 align:middle line:84% The peaks and shapes are significant. 00:20:33.420 --> 00:20:34.800 align:middle line:90% Yep. 00:20:34.800 --> 00:20:35.607 align:middle line:90% You have to step-- 00:20:35.607 --> 00:20:37.440 align:middle line:84% oh, Gary has to step away for a few minutes. 00:20:37.440 --> 00:20:40.740 align:middle line:84% Not an accessibility issue, Gary, but I'll accept it. 00:20:40.740 --> 00:20:42.870 align:middle line:84% Need some way to interactively explore graphs 00:20:42.870 --> 00:20:44.020 align:middle line:90% and read off chosen points. 00:20:44.020 --> 00:20:44.520 align:middle line:90% Yeah. 00:20:44.520 --> 00:20:46.800 align:middle line:84% All these things are currently true, 00:20:46.800 --> 00:20:49.440 align:middle line:90% so why don't we stop there. 00:20:49.440 --> 00:20:49.940 align:middle line:90% OK. 00:20:49.940 --> 00:20:51.420 align:middle line:90% And I'll add just a few others. 00:20:51.420 --> 00:20:53.420 align:middle line:84% One of the challenges with accessibility, 00:20:53.420 --> 00:20:56.390 align:middle line:84% oftentimes, is we have this sort of knee-jerk reaction 00:20:56.390 --> 00:21:00.230 align:middle line:84% to say, well, let's just put an alternate description 00:21:00.230 --> 00:21:02.030 align:middle line:90% in our website or something. 00:21:02.030 --> 00:21:04.880 align:middle line:84% And an alternate description is absolutely better than nothing. 00:21:04.880 --> 00:21:06.950 align:middle line:84% I mean, not having any information at all 00:21:06.950 --> 00:21:07.640 align:middle line:90% is clearly bad. 00:21:07.640 --> 00:21:09.890 align:middle line:84% But the problem is the alternate description 00:21:09.890 --> 00:21:12.140 align:middle line:84% makes the presumption that the person that 00:21:12.140 --> 00:21:14.900 align:middle line:84% wrote the description actually is telling you 00:21:14.900 --> 00:21:17.660 align:middle line:84% the information that you want as a user, 00:21:17.660 --> 00:21:19.165 align:middle line:84% and that may not be the case, right? 00:21:19.165 --> 00:21:21.540 align:middle line:84% You might want all sorts of information about this graph. 00:21:21.540 --> 00:21:23.660 align:middle line:84% You might be looking for what's the scale, 00:21:23.660 --> 00:21:25.730 align:middle line:84% and that wasn't in the alternate description. 00:21:25.730 --> 00:21:28.460 align:middle line:84% You might be looking for when are the blips? 00:21:28.460 --> 00:21:30.350 align:middle line:84% What is the periodic pattern of the blips? 00:21:30.350 --> 00:21:32.180 align:middle line:84% Or any other number of other things 00:21:32.180 --> 00:21:34.790 align:middle line:84% depending on what field of study you're in. 00:21:34.790 --> 00:21:36.560 align:middle line:84% And I'll tell you, having collaborated 00:21:36.560 --> 00:21:38.600 align:middle line:90% with groups in other fields-- 00:21:38.600 --> 00:21:43.170 align:middle line:84% one of our projects is in making astronomy more accessible. 00:21:43.170 --> 00:21:46.190 align:middle line:84% The kinds of information that they look for in graphs 00:21:46.190 --> 00:21:49.940 align:middle line:84% is very different than when I run rat-controlled trials 00:21:49.940 --> 00:21:51.890 align:middle line:84% on programming languages or other topics. 00:21:51.890 --> 00:21:54.420 align:middle line:84% It's just the information we're looking for on these charts 00:21:54.420 --> 00:21:56.860 align:middle line:90% is very different. 00:21:56.860 --> 00:21:59.890 align:middle line:84% No auditory description would be good enough because there's 00:21:59.890 --> 00:22:03.190 align:middle line:90% other things we're looking for. 00:22:03.190 --> 00:22:06.300 align:middle line:84% So I kind of wrote these up as a roundup in the slides. 00:22:06.300 --> 00:22:09.510 align:middle line:84% And I'll just briefly touch on them. 00:22:09.510 --> 00:22:11.220 align:middle line:90% So library designs. 00:22:11.220 --> 00:22:13.800 align:middle line:84% And basically every statistical language that I've looked at 00:22:13.800 --> 00:22:16.410 align:middle line:84% have poorly thought out acronyms. 00:22:16.410 --> 00:22:18.317 align:middle line:84% Functions named after mathematicians. 00:22:18.317 --> 00:22:20.150 align:middle line:84% Really shouldn't be called Crucible Wallace. 00:22:20.150 --> 00:22:22.320 align:middle line:84% It should have a meaningful name, right? 00:22:22.320 --> 00:22:23.730 align:middle line:90% Single letter label names. 00:22:23.730 --> 00:22:27.120 align:middle line:84% F-test, t-test, p-value, b-distribution, d-distribution, 00:22:27.120 --> 00:22:28.990 align:middle line:84% gamma distribution, all this kind of stuff. 00:22:28.990 --> 00:22:30.570 align:middle line:90% They make no sense. 00:22:30.570 --> 00:22:32.340 align:middle line:84% Names that lack semantic meaning, 00:22:32.340 --> 00:22:34.690 align:middle line:84% like anything with the words gg in front of it, 00:22:34.690 --> 00:22:36.660 align:middle line:84% regardless of the library that is. 00:22:36.660 --> 00:22:38.610 align:middle line:90% The tilde operator. 00:22:38.610 --> 00:22:40.523 align:middle line:90% No global organization system. 00:22:40.523 --> 00:22:41.940 align:middle line:84% Oftentimes, these things have been 00:22:41.940 --> 00:22:43.410 align:middle line:90% built by giant communities. 00:22:43.410 --> 00:22:44.340 align:middle line:90% And that makes sense. 00:22:44.340 --> 00:22:46.710 align:middle line:84% That's how you get stuff built. But they 00:22:46.710 --> 00:22:50.160 align:middle line:84% lack any kind of like, hey, does your name make any sense? 00:22:50.160 --> 00:22:52.980 align:middle line:84% Hey, what's the evidence for your naming convention? 00:22:52.980 --> 00:22:55.620 align:middle line:84% Or a global organization is just totally missing 00:22:55.620 --> 00:22:57.710 align:middle line:90% in data science, I think. 00:22:57.710 --> 00:22:59.600 align:middle line:84% And that's not an easy problem to solve 00:22:59.600 --> 00:23:00.600 align:middle line:90% nor is it an indictment. 00:23:00.600 --> 00:23:02.250 align:middle line:84% It's not like we all go and say, aha! 00:23:02.250 --> 00:23:03.750 align:middle line:84% We'll wave the magic wand and then 00:23:03.750 --> 00:23:05.310 align:middle line:90% we'll all be super organized. 00:23:05.310 --> 00:23:07.230 align:middle line:84% It doesn't work that way back in reality. 00:23:07.230 --> 00:23:11.100 align:middle line:84% But nonetheless, it is true that it lacks a global organization 00:23:11.100 --> 00:23:13.620 align:middle line:90% and naming system, right? 00:23:13.620 --> 00:23:17.430 align:middle line:84% So also it's not just the languages, though. 00:23:17.430 --> 00:23:19.920 align:middle line:84% Programming languages, in many of these fields, 00:23:19.920 --> 00:23:24.090 align:middle line:84% they actually lack accessibility support at the language level. 00:23:24.090 --> 00:23:26.130 align:middle line:84% And these have very specific libraries. 00:23:26.130 --> 00:23:28.200 align:middle line:84% For example, in Windows, it's called UIA. 00:23:28.200 --> 00:23:29.820 align:middle line:90% I'll talk about that later. 00:23:29.820 --> 00:23:32.040 align:middle line:84% In Apple, it's called NSAccessibility. 00:23:32.040 --> 00:23:34.343 align:middle line:84% In Android, they have a Talkback API. 00:23:34.343 --> 00:23:35.760 align:middle line:84% And on the web, they often call it 00:23:35.760 --> 00:23:38.100 align:middle line:84% ARIA related to this thing called WCAG, 00:23:38.100 --> 00:23:41.460 align:middle line:84% but actually, those just connect to UIA and NSAccessibility, 00:23:41.460 --> 00:23:43.410 align:middle line:90% so it's complicated. 00:23:43.410 --> 00:23:46.530 align:middle line:84% But when this support is lacking, 00:23:46.530 --> 00:23:48.480 align:middle line:84% that means any tool that you build 00:23:48.480 --> 00:23:50.310 align:middle line:84% on top of those programming languages 00:23:50.310 --> 00:23:52.530 align:middle line:90% is automatically not accessible. 00:23:52.530 --> 00:23:54.150 align:middle line:84% Just out of the gate, there's nothing 00:23:54.150 --> 00:23:56.770 align:middle line:84% you can do about it unless you add those libraries. 00:23:56.770 --> 00:23:59.160 align:middle line:84% And I'll talk more about why that is and what that is. 00:23:59.160 --> 00:24:02.003 align:middle line:84% But a couple more in terms of accessibility problems. 00:24:02.003 --> 00:24:03.420 align:middle line:84% Oftentimes, there's input problems 00:24:03.420 --> 00:24:06.580 align:middle line:84% like no or poor keyboard support, no virtual keyboard 00:24:06.580 --> 00:24:07.080 align:middle line:90% support. 00:24:07.080 --> 00:24:08.205 align:middle line:90% This actually sounds weird. 00:24:08.205 --> 00:24:10.590 align:middle line:84% A virtual keyboard is not your physical touching thing. 00:24:10.590 --> 00:24:12.840 align:middle line:84% Virtual keyboards actually work a little bit different 00:24:12.840 --> 00:24:15.215 align:middle line:84% and it's really easy to screw up virtual keyboard support 00:24:15.215 --> 00:24:16.560 align:middle line:90% in accessibility technologies. 00:24:16.560 --> 00:24:19.090 align:middle line:84% I've screwed it up myself before. 00:24:19.090 --> 00:24:21.090 align:middle line:84% And then there's lots and lots of issues related 00:24:21.090 --> 00:24:24.020 align:middle line:90% to graphics and structure. 00:24:24.020 --> 00:24:27.590 align:middle line:84% No resizing of text or images, no alternate descriptions, 00:24:27.590 --> 00:24:29.860 align:middle line:84% graphs render without explorable structure, 00:24:29.860 --> 00:24:32.570 align:middle line:84% unformatted raw output with really silly naming 00:24:32.570 --> 00:24:35.000 align:middle line:84% conventions, inaccessible PDFs that get 00:24:35.000 --> 00:24:37.970 align:middle line:84% generated, videos that lack closed captioning, 00:24:37.970 --> 00:24:39.320 align:middle line:90% all sorts of stuff like that. 00:24:39.320 --> 00:24:42.380 align:middle line:84% The point isn't that this is a total list of all 00:24:42.380 --> 00:24:44.060 align:middle line:84% the accessibility issues, I'm just 00:24:44.060 --> 00:24:47.240 align:middle line:84% trying to give you a flavor of the types of things 00:24:47.240 --> 00:24:50.420 align:middle line:84% that I see when I do accessibility analysis on tops 00:24:50.420 --> 00:24:52.868 align:middle line:84% of all the tools that I mentioned previously. 00:24:52.868 --> 00:24:54.410 align:middle line:84% These are the kinds of things that we 00:24:54.410 --> 00:24:58.310 align:middle line:90% observe on a factual basis. 00:24:58.310 --> 00:24:59.390 align:middle line:90% OK, cool. 00:24:59.390 --> 00:25:03.077 align:middle line:84% So I just told you about the billion hours of work 00:25:03.077 --> 00:25:04.910 align:middle line:84% that would be needed to fix all these issues 00:25:04.910 --> 00:25:08.780 align:middle line:84% across these tools, but how do you even do it? 00:25:08.780 --> 00:25:12.320 align:middle line:84% Because it turns out that accessibility and testing 00:25:12.320 --> 00:25:16.130 align:middle line:84% for it is kind of complicated and it's not super easy. 00:25:16.130 --> 00:25:21.110 align:middle line:84% And in fact, because the API differ across every platform, 00:25:21.110 --> 00:25:24.260 align:middle line:84% I really can't, in one hour, describe how 00:25:24.260 --> 00:25:25.520 align:middle line:90% this works for every platform. 00:25:25.520 --> 00:25:27.290 align:middle line:90% It's just not possible. 00:25:27.290 --> 00:25:30.470 align:middle line:84% These platforms are incompatible and accessibility 00:25:30.470 --> 00:25:34.850 align:middle line:84% is 0% compatible with UI Automation on Windows, 00:25:34.850 --> 00:25:38.640 align:middle line:84% and same with the Talkback API and similar stuff, as well. 00:25:38.640 --> 00:25:40.268 align:middle line:84% So what I'm going to do is I'm just 00:25:40.268 --> 00:25:41.810 align:middle line:84% going to talk a little bit more today 00:25:41.810 --> 00:25:44.780 align:middle line:84% about one of these platforms to give you a flavor for how 00:25:44.780 --> 00:25:46.550 align:middle line:90% they work across each. 00:25:46.550 --> 00:25:48.590 align:middle line:84% And then that will hopefully give you 00:25:48.590 --> 00:25:51.410 align:middle line:84% a sense of how you do testing, at least on one platform. 00:25:51.410 --> 00:25:53.530 align:middle line:84% And it's different on the others. 00:25:53.530 --> 00:25:56.690 align:middle line:84% So what I'm going to focus on is called UI Automation 00:25:56.690 --> 00:25:58.325 align:middle line:90% on specifically Windows. 00:25:58.325 --> 00:26:00.200 align:middle line:84% And I'm doing that one in part just because I 00:26:00.200 --> 00:26:03.440 align:middle line:84% know it the best, but also because it's really mature, 00:26:03.440 --> 00:26:06.248 align:middle line:84% and so you can do accessibility black magic with it if you 00:26:06.248 --> 00:26:07.790 align:middle line:84% know what you're doing, which is neat. 00:26:07.790 --> 00:26:10.510 align:middle line:90% 00:26:10.510 --> 00:26:11.920 align:middle line:90% So what is UI Automation? 00:26:11.920 --> 00:26:15.490 align:middle line:84% Well, it's a low level, C or C++, 00:26:15.490 --> 00:26:18.520 align:middle line:84% way to interact with the operating system 00:26:18.520 --> 00:26:21.280 align:middle line:90% for specifically accessibility. 00:26:21.280 --> 00:26:22.870 align:middle line:90% It's low level. 00:26:22.870 --> 00:26:27.250 align:middle line:84% It's really, really gross, but it is functional. 00:26:27.250 --> 00:26:30.940 align:middle line:84% And if you want any programming language at all 00:26:30.940 --> 00:26:33.820 align:middle line:84% to have accessibility support on Windows specifically, 00:26:33.820 --> 00:26:36.770 align:middle line:84% especially when Windows 10 it's kind of the only game in town. 00:26:36.770 --> 00:26:39.178 align:middle line:84% There are other historical techniques 00:26:39.178 --> 00:26:40.720 align:middle line:84% that have been used for accessibility 00:26:40.720 --> 00:26:44.770 align:middle line:84% on these platforms, but UIA is the solution today 00:26:44.770 --> 00:26:47.830 align:middle line:90% for doing accessibility work. 00:26:47.830 --> 00:26:50.560 align:middle line:84% And UIA has two things you have to know about it. 00:26:50.560 --> 00:26:55.070 align:middle line:84% It has clients and it has providers, OK? 00:26:55.070 --> 00:26:57.670 align:middle line:84% And this is slightly tricky, but the core lesson 00:26:57.670 --> 00:27:00.640 align:middle line:84% is products made in programming languages that 00:27:00.640 --> 00:27:03.100 align:middle line:84% lack this mapping on these operating systems 00:27:03.100 --> 00:27:05.500 align:middle line:84% are automatically not accessible. 00:27:05.500 --> 00:27:07.930 align:middle line:84% You just-- out of the gate, they won't work, 00:27:07.930 --> 00:27:10.318 align:middle line:84% or they'll work very poorly for people with disabilities 00:27:10.318 --> 00:27:11.110 align:middle line:90% on a lot of fronts. 00:27:11.110 --> 00:27:13.420 align:middle line:84% And I'll kind of explain why that is. 00:27:13.420 --> 00:27:16.030 align:middle line:84% So first, let's talk about clients. 00:27:16.030 --> 00:27:20.920 align:middle line:84% When UIA clients access information 00:27:20.920 --> 00:27:23.750 align:middle line:84% from the operating system, it wouldn't make sense for them 00:27:23.750 --> 00:27:25.750 align:middle line:84% to have to go to every single application that's 00:27:25.750 --> 00:27:28.527 align:middle line:84% ever written on Earth and say, hey, what do you provide? 00:27:28.527 --> 00:27:29.360 align:middle line:90% What do you provide? 00:27:29.360 --> 00:27:30.193 align:middle line:90% What do you provide? 00:27:30.193 --> 00:27:32.950 align:middle line:84% That would be too difficult. So what they do instead 00:27:32.950 --> 00:27:36.750 align:middle line:84% is they connect down to UIA and ask any application that 00:27:36.750 --> 00:27:41.100 align:middle line:84% is provided the correct mappings, what do they support, 00:27:41.100 --> 00:27:41.800 align:middle line:90% right? 00:27:41.800 --> 00:27:44.510 align:middle line:84% And so these applications vary a lot. 00:27:44.510 --> 00:27:46.625 align:middle line:84% So the classic one is a screen reader that-- 00:27:46.625 --> 00:27:47.750 align:middle line:90% people know about that one. 00:27:47.750 --> 00:27:50.230 align:middle line:84% But people often don't realize when Microsoft 00:27:50.230 --> 00:27:52.720 align:middle line:84% made accessibility support, they didn't 00:27:52.720 --> 00:27:56.110 align:middle line:84% make screen reader support and then virtual keyboard support 00:27:56.110 --> 00:27:58.570 align:middle line:84% and then magnifier support and stuff like that. 00:27:58.570 --> 00:28:00.700 align:middle line:84% They all run through the same architecture 00:28:00.700 --> 00:28:02.560 align:middle line:84% because they don't want to go crazy, right? 00:28:02.560 --> 00:28:04.880 align:middle line:84% It's just simpler to do it that way. 00:28:04.880 --> 00:28:08.410 align:middle line:84% So when you break something for, let's say, a screen reader, you 00:28:08.410 --> 00:28:11.080 align:middle line:84% might actually break it across a whole range 00:28:11.080 --> 00:28:13.090 align:middle line:84% of accessibility products, right? 00:28:13.090 --> 00:28:15.130 align:middle line:84% Virtual keyboards, sip and puff readers, 00:28:15.130 --> 00:28:18.010 align:middle line:84% which are on the screen, screen readers, Braille devices, 00:28:18.010 --> 00:28:19.390 align:middle line:90% magnifiers, all sorts of stuff. 00:28:19.390 --> 00:28:21.460 align:middle line:84% They all use similar technologies under the hood 00:28:21.460 --> 00:28:24.530 align:middle line:90% through UIA. 00:28:24.530 --> 00:28:25.820 align:middle line:90% So that's the idea. 00:28:25.820 --> 00:28:29.660 align:middle line:84% And they look at clients, look at information 00:28:29.660 --> 00:28:32.870 align:middle line:84% provided through UIA through event structures and structure. 00:28:32.870 --> 00:28:35.630 align:middle line:84% So if it's a screen reader, it connects UIA, 00:28:35.630 --> 00:28:37.940 align:middle line:84% it gets its structure, and it listens to its events, 00:28:37.940 --> 00:28:41.330 align:middle line:84% and then it responds accordingly, right? 00:28:41.330 --> 00:28:42.320 align:middle line:90% So lesson two. 00:28:42.320 --> 00:28:46.340 align:middle line:84% Clients consume information across all disability types 00:28:46.340 --> 00:28:47.510 align:middle line:90% with the same architecture. 00:28:47.510 --> 00:28:49.060 align:middle line:84% And on Windows, that's called UIA. 00:28:49.060 --> 00:28:52.490 align:middle line:90% On Mac, it's NSAccessibility. 00:28:52.490 --> 00:28:54.600 align:middle line:90% And UIA also has providers. 00:28:54.600 --> 00:28:57.990 align:middle line:84% And providers, that's what individual applications have 00:28:57.990 --> 00:29:00.890 align:middle line:84% to write in order for their content to be accessible. 00:29:00.890 --> 00:29:03.870 align:middle line:84% So if you're making, hypothetically, RStudio, you 00:29:03.870 --> 00:29:06.810 align:middle line:84% need to have a provider that sends information 00:29:06.810 --> 00:29:09.150 align:middle line:84% to the system for your application. 00:29:09.150 --> 00:29:10.900 align:middle line:84% And that may or may not be easy to do, 00:29:10.900 --> 00:29:12.260 align:middle line:84% depending upon the programming language. 00:29:12.260 --> 00:29:13.650 align:middle line:90% It's not flip a switch, right? 00:29:13.650 --> 00:29:15.990 align:middle line:90% It's not simple. 00:29:15.990 --> 00:29:19.060 align:middle line:84% So what providers do, they provide three things. 00:29:19.060 --> 00:29:24.840 align:middle line:84% One is control patterns, one is properties, and one is events. 00:29:24.840 --> 00:29:27.220 align:middle line:84% Those three are critically, critically important. 00:29:27.220 --> 00:29:28.860 align:middle line:84% I'm going to show you a couple of them. 00:29:28.860 --> 00:29:30.690 align:middle line:84% So sometimes when people email, they 00:29:30.690 --> 00:29:32.220 align:middle line:84% ask why is the Quorum programming 00:29:32.220 --> 00:29:33.070 align:middle line:90% language accessible? 00:29:33.070 --> 00:29:34.945 align:middle line:84% It's because we have a really fancy provider. 00:29:34.945 --> 00:29:36.450 align:middle line:84% That's really what it comes down to. 00:29:36.450 --> 00:29:38.340 align:middle line:84% Quorum has a really fancy provider 00:29:38.340 --> 00:29:41.027 align:middle line:84% that can do stuff for all sorts of kinds of applications, 00:29:41.027 --> 00:29:41.610 align:middle line:90% and that's it. 00:29:41.610 --> 00:29:43.780 align:middle line:84% That's really what it comes down to. 00:29:43.780 --> 00:29:45.550 align:middle line:90% OK, lesson 3. 00:29:45.550 --> 00:29:47.963 align:middle line:84% If you don't implement a provider or one is inside 00:29:47.963 --> 00:29:49.630 align:middle line:84% of the programming language you're using 00:29:49.630 --> 00:29:51.460 align:middle line:84% or it's a bad provider, as happens, 00:29:51.460 --> 00:29:54.180 align:middle line:84% Java has kind of really terrible provider, 00:29:54.180 --> 00:29:57.670 align:middle line:84% your application, anything built on top of that, 00:29:57.670 --> 00:29:59.320 align:middle line:84% is not going to be accessible or it's 00:29:59.320 --> 00:30:00.850 align:middle line:90% going to be poorly accessible. 00:30:00.850 --> 00:30:02.500 align:middle line:84% So let's look at a couple of these. 00:30:02.500 --> 00:30:03.550 align:middle line:84% And again, I'm going to share a screen 00:30:03.550 --> 00:30:05.508 align:middle line:84% in a different way for this so you can actually 00:30:05.508 --> 00:30:08.080 align:middle line:90% see some of these. 00:30:08.080 --> 00:30:09.740 align:middle line:84% They're not that complicated, but I 00:30:09.740 --> 00:30:11.730 align:middle line:84% have to share them in a different way. 00:30:11.730 --> 00:30:14.230 align:middle line:84% So first what I'm going to do is I'm going to show you a low 00:30:14.230 --> 00:30:15.590 align:middle line:90% level C++ provider for a button. 00:30:15.590 --> 00:30:19.050 align:middle line:84% I'm showing you a button because it's by far the simplest thing. 00:30:19.050 --> 00:30:22.130 align:middle line:84% And you'll notice that this is low level C++. 00:30:22.130 --> 00:30:24.350 align:middle line:84% And if you're already thinking, oh, dear God, Stefik, 00:30:24.350 --> 00:30:26.180 align:middle line:84% I'm going to vomit, then you're right. 00:30:26.180 --> 00:30:29.960 align:middle line:84% That's the correct answer and you should know that. 00:30:29.960 --> 00:30:32.960 align:middle line:84% But it's the abstraction that we have, 00:30:32.960 --> 00:30:35.750 align:middle line:84% so you just kind of have to deal with it. 00:30:35.750 --> 00:30:37.040 align:middle line:90% It is the reality. 00:30:37.040 --> 00:30:39.710 align:middle line:84% And also the problem UIA is solving 00:30:39.710 --> 00:30:41.370 align:middle line:90% is actually very complex. 00:30:41.370 --> 00:30:43.100 align:middle line:84% And so I have to give Microsoft props 00:30:43.100 --> 00:30:45.710 align:middle line:84% that even though the abstraction is kind of gross-looking, 00:30:45.710 --> 00:30:48.500 align:middle line:84% it does work, and that's important, right? 00:30:48.500 --> 00:30:51.112 align:middle line:84% It does actually work across these products. 00:30:51.112 --> 00:30:53.070 align:middle line:84% OK, now I'm not going to read this code to you, 00:30:53.070 --> 00:30:54.830 align:middle line:84% but I do want to point out a couple of things. 00:30:54.830 --> 00:30:56.247 align:middle line:84% And they'll come back later, which 00:30:56.247 --> 00:30:58.790 align:middle line:84% is why I bothered to show you some source code. 00:30:58.790 --> 00:31:01.160 align:middle line:84% Inside of this funny thing, I have a thing 00:31:01.160 --> 00:31:02.920 align:middle line:90% called a button provider. 00:31:02.920 --> 00:31:07.130 align:middle line:84% And inside of it, it implements this thing called an IInvoke 00:31:07.130 --> 00:31:08.090 align:middle line:90% provider. 00:31:08.090 --> 00:31:11.240 align:middle line:84% And you might think, dude, that's super low level 00:31:11.240 --> 00:31:11.840 align:middle line:90% and detailed. 00:31:11.840 --> 00:31:12.965 align:middle line:90% Why do I need to know that? 00:31:12.965 --> 00:31:15.770 align:middle line:84% And the reason is because when we evaluate 00:31:15.770 --> 00:31:17.940 align:middle line:84% accessibility a little bit later, 00:31:17.940 --> 00:31:19.730 align:middle line:84% these are the same names that show up 00:31:19.730 --> 00:31:21.270 align:middle line:90% in the accessibility tools. 00:31:21.270 --> 00:31:23.360 align:middle line:84% So if you don't know the low level names, 00:31:23.360 --> 00:31:25.110 align:middle line:84% you won't understand what it's telling you 00:31:25.110 --> 00:31:27.693 align:middle line:84% when you get back accessibility information from the operating 00:31:27.693 --> 00:31:28.340 align:middle line:90% system. 00:31:28.340 --> 00:31:30.260 align:middle line:84% Unfortunately, they're linked and they're not 00:31:30.260 --> 00:31:31.610 align:middle line:90% translated into English. 00:31:31.610 --> 00:31:33.750 align:middle line:90% It just is what it is. 00:31:33.750 --> 00:31:35.960 align:middle line:84% So there's this thing called IInvoke provider that's 00:31:35.960 --> 00:31:38.780 align:middle line:84% basically telling the operating system, hey, I have a thing 00:31:38.780 --> 00:31:41.630 align:middle line:84% and I'm declaring to you that it's a button. 00:31:41.630 --> 00:31:43.060 align:middle line:90% What is it on screen? 00:31:43.060 --> 00:31:44.060 align:middle line:90% Doesn't matter. 00:31:44.060 --> 00:31:44.990 align:middle line:90% Spinning dragon? 00:31:44.990 --> 00:31:45.710 align:middle line:90% Cool. 00:31:45.710 --> 00:31:47.240 align:middle line:90% It's a spinning dragon button. 00:31:47.240 --> 00:31:48.110 align:middle line:90% Right on. 00:31:48.110 --> 00:31:50.720 align:middle line:90% Is it a fire-breathing monkey? 00:31:50.720 --> 00:31:52.000 align:middle line:90% Of course it is. 00:31:52.000 --> 00:31:52.760 align:middle line:90% Fine. 00:31:52.760 --> 00:31:55.355 align:middle line:84% It's a fire-breathing monkey, but it can be invoked. 00:31:55.355 --> 00:31:55.980 align:middle line:90% Doesn't matter. 00:31:55.980 --> 00:31:57.260 align:middle line:90% But that's the cool thing. 00:31:57.260 --> 00:31:58.158 align:middle line:90% You can declare. 00:31:58.158 --> 00:32:00.200 align:middle line:84% And this is how you can make things like graphics 00:32:00.200 --> 00:32:00.700 align:middle line:90% accessible. 00:32:00.700 --> 00:32:02.960 align:middle line:90% Hopefully that's a hint, right? 00:32:02.960 --> 00:32:04.410 align:middle line:90% But there's some other stuff. 00:32:04.410 --> 00:32:07.007 align:middle line:84% One is there's this thing called, IsPatternSupported. 00:32:07.007 --> 00:32:09.590 align:middle line:84% This is kind of a thing that we do in the Quorum architecture. 00:32:09.590 --> 00:32:11.090 align:middle line:84% And I'll get to what that is once we 00:32:11.090 --> 00:32:12.650 align:middle line:90% get to the implementation. 00:32:12.650 --> 00:32:15.140 align:middle line:84% And then there's this thing that says getControlType which 00:32:15.140 --> 00:32:19.310 align:middle line:84% is really, really important because it has to do with how 00:32:19.310 --> 00:32:21.410 align:middle line:90% things interact with your tool. 00:32:21.410 --> 00:32:24.110 align:middle line:84% And then finally, there's this thing, and I love the name. 00:32:24.110 --> 00:32:26.510 align:middle line:84% It makes perfect sense, obviously, it's very intuitive, 00:32:26.510 --> 00:32:28.700 align:middle line:90% called IFACEMETHODIMP, right? 00:32:28.700 --> 00:32:31.000 align:middle line:84% Obviously, I always think of a garden gnome 00:32:31.000 --> 00:32:32.250 align:middle line:90% that's, like, sitting around-- 00:32:32.250 --> 00:32:33.140 align:middle line:90% I don't know. 00:32:33.140 --> 00:32:35.540 align:middle line:84% This is saying this is an interface method 00:32:35.540 --> 00:32:37.820 align:middle line:84% and it's an implementation of it. 00:32:37.820 --> 00:32:41.090 align:middle line:84% It's a convention in Windows programming. 00:32:41.090 --> 00:32:42.020 align:middle line:90% I'm sorry. 00:32:42.020 --> 00:32:43.520 align:middle line:90% And it says invoke. 00:32:43.520 --> 00:32:47.270 align:middle line:84% This is saying, if you have that spinning, fire-breathing monkey 00:32:47.270 --> 00:32:50.390 align:middle line:84% on the screen, and the AT technology, no matter 00:32:50.390 --> 00:32:53.090 align:middle line:84% what kind it is, wants to tell it to invoke it, 00:32:53.090 --> 00:32:54.650 align:middle line:84% to click the button without actually 00:32:54.650 --> 00:32:58.010 align:middle line:84% pressing a key or something with that, they can do that. 00:32:58.010 --> 00:33:00.110 align:middle line:84% And this then triggers an event that sends it up 00:33:00.110 --> 00:33:01.010 align:middle line:90% to your system. 00:33:01.010 --> 00:33:02.360 align:middle line:90% It's sort of like-- 00:33:02.360 --> 00:33:04.160 align:middle line:84% people with different kinds of disabilities 00:33:04.160 --> 00:33:06.620 align:middle line:84% interact with computers in all sorts of ways. 00:33:06.620 --> 00:33:09.320 align:middle line:84% And they can declare to the operating system 00:33:09.320 --> 00:33:10.740 align:middle line:84% that they want to invoke a button, 00:33:10.740 --> 00:33:12.580 align:middle line:84% even though they may not be physically pressing the key. 00:33:12.580 --> 00:33:13.820 align:middle line:84% They might have a sip and puff, they 00:33:13.820 --> 00:33:15.500 align:middle line:84% might have any number of other technologies 00:33:15.500 --> 00:33:16.875 align:middle line:84% that they're using, and you can't 00:33:16.875 --> 00:33:18.740 align:middle line:84% know as the application developer what 00:33:18.740 --> 00:33:22.910 align:middle line:84% that is because it's however they choose. 00:33:22.910 --> 00:33:23.410 align:middle line:90% OK. 00:33:23.410 --> 00:33:24.850 align:middle line:84% So then there's an implementation. 00:33:24.850 --> 00:33:26.170 align:middle line:84% And I'll show you this really quick. 00:33:26.170 --> 00:33:27.462 align:middle line:90% It's really not that much code. 00:33:27.462 --> 00:33:30.520 align:middle line:84% It's relatively small, but it's esoteric. 00:33:30.520 --> 00:33:32.530 align:middle line:90% And that's the weird part. 00:33:32.530 --> 00:33:33.963 align:middle line:84% So I have this IsPatternSupported, 00:33:33.963 --> 00:33:36.130 align:middle line:84% and you'll notice that there's this funny thing that 00:33:36.130 --> 00:33:40.450 align:middle line:84% says, if the pattern ID provided by the operating system 00:33:40.450 --> 00:33:45.550 align:middle line:84% matches UIA invoke pattern ID, that it returns true. 00:33:45.550 --> 00:33:47.620 align:middle line:84% And this sounds bonkers, but what 00:33:47.620 --> 00:33:50.920 align:middle line:84% it's telling you is it's saying, I am a button. 00:33:50.920 --> 00:33:52.270 align:middle line:90% That's what this is saying. 00:33:52.270 --> 00:33:55.780 align:middle line:84% And when we pull up in a moment the accessibility checkers, 00:33:55.780 --> 00:33:57.710 align:middle line:84% you will literally see a thing that says, hey, 00:33:57.710 --> 00:33:58.590 align:middle line:90% this is a button. 00:33:58.590 --> 00:34:00.760 align:middle line:84% And it will tell you this pattern is provided. 00:34:00.760 --> 00:34:02.483 align:middle line:90% That's why I'm showing you this. 00:34:02.483 --> 00:34:04.150 align:middle line:84% And then in addition, there's this thing 00:34:04.150 --> 00:34:05.983 align:middle line:84% that says the control type, and this is just 00:34:05.983 --> 00:34:09.850 align:middle line:84% a number under the hood, is a UIA button control type ID. 00:34:09.850 --> 00:34:11.739 align:middle line:90% This is critically important. 00:34:11.739 --> 00:34:14.020 align:middle line:84% And your accessibility will be completely broken 00:34:14.020 --> 00:34:17.139 align:middle line:84% if this is wrong because this tells something like a screen 00:34:17.139 --> 00:34:21.100 align:middle line:84% reader or other device what kind of thing it is, which tells it 00:34:21.100 --> 00:34:23.300 align:middle line:90% how it reads it to the user. 00:34:23.300 --> 00:34:24.940 align:middle line:84% So it says, oh, this is a button? 00:34:24.940 --> 00:34:29.350 align:middle line:84% Tell the user this is a button verbally, or whatever, right? 00:34:29.350 --> 00:34:31.788 align:middle line:84% And then finally, this invoke NR system, this 00:34:31.788 --> 00:34:33.580 align:middle line:84% won't mean anything to anybody, but it just 00:34:33.580 --> 00:34:36.070 align:middle line:84% passes an invoke event back to us. 00:34:36.070 --> 00:34:37.030 align:middle line:90% OK. 00:34:37.030 --> 00:34:39.880 align:middle line:84% Now I want to show you one more of these providers, 00:34:39.880 --> 00:34:42.510 align:middle line:84% just so we can get a slightly hard one. 00:34:42.510 --> 00:34:44.469 align:middle line:90% And so let's pull up a checkbox. 00:34:44.469 --> 00:34:48.120 align:middle line:84% And I'll go a little bit faster for this one. 00:34:48.120 --> 00:34:51.030 align:middle line:84% In a checkbox, the important thing is instead 00:34:51.030 --> 00:34:53.850 align:middle line:84% of an IInvokeProvider, I've got an IToggleProvider 00:34:53.850 --> 00:34:56.280 align:middle line:84% because it toggles things on and off. 00:34:56.280 --> 00:34:58.780 align:middle line:84% And then I've still got the same two methods, 00:34:58.780 --> 00:35:00.540 align:middle line:84% whether it supports a pattern or not, 00:35:00.540 --> 00:35:03.000 align:middle line:84% And then I've got a function to toggle and a function 00:35:03.000 --> 00:35:03.930 align:middle line:90% to get toggle state. 00:35:03.930 --> 00:35:05.130 align:middle line:84% And I'll show you why this matters 00:35:05.130 --> 00:35:07.172 align:middle line:84% in just a second, because it tends to do with how 00:35:07.172 --> 00:35:08.400 align:middle line:90% it sends information around. 00:35:08.400 --> 00:35:11.550 align:middle line:90% 00:35:11.550 --> 00:35:14.380 align:middle line:84% So I'm going to pull up the implementation here. 00:35:14.380 --> 00:35:17.090 align:middle line:84% And in this particular case, my pattern 00:35:17.090 --> 00:35:20.240 align:middle line:90% is a UIA toggle pattern ID. 00:35:20.240 --> 00:35:22.130 align:middle line:84% That tells the operating system how it 00:35:22.130 --> 00:35:23.900 align:middle line:90% can be interacted with, right? 00:35:23.900 --> 00:35:26.750 align:middle line:84% What commands can be sent back and forth. 00:35:26.750 --> 00:35:30.530 align:middle line:84% And then I've got a UIA checkbox control type ID. 00:35:30.530 --> 00:35:32.720 align:middle line:84% This tells the operating system, hey, 00:35:32.720 --> 00:35:35.180 align:middle line:84% this spinning, fire-breathing monkey can 00:35:35.180 --> 00:35:36.890 align:middle line:90% be flipped on and off, right? 00:35:36.890 --> 00:35:38.720 align:middle line:84% Again, the graphical representation 00:35:38.720 --> 00:35:41.840 align:middle line:84% is entirely irrelevant to the operating system. 00:35:41.840 --> 00:35:44.060 align:middle line:84% All it cares about is what kind of thing 00:35:44.060 --> 00:35:45.710 align:middle line:90% am I declaring that this is. 00:35:45.710 --> 00:35:47.310 align:middle line:90% I could be completely lying. 00:35:47.310 --> 00:35:50.180 align:middle line:84% And in fact, in Quorum, we repeatedly and consistently lie 00:35:50.180 --> 00:35:52.220 align:middle line:84% to the operating system all the time. 00:35:52.220 --> 00:35:54.060 align:middle line:90% That's just normal. 00:35:54.060 --> 00:35:54.560 align:middle line:90% OK. 00:35:54.560 --> 00:35:56.102 align:middle line:84% The last thing I want to mention here 00:35:56.102 --> 00:35:57.950 align:middle line:84% is when I actually have a toggle, 00:35:57.950 --> 00:35:59.660 align:middle line:84% you have to do some weird black magic. 00:35:59.660 --> 00:36:01.327 align:middle line:84% And I mention these things because there 00:36:01.327 --> 00:36:04.160 align:middle line:84% are certain things in UIA that are kind of broken, 00:36:04.160 --> 00:36:06.110 align:middle line:84% and you should just be aware that when you're 00:36:06.110 --> 00:36:08.130 align:middle line:84% getting inconsistent screen reader behavior, 00:36:08.130 --> 00:36:09.980 align:middle line:90% this is why it happens. 00:36:09.980 --> 00:36:12.920 align:middle line:84% Notably, screen readers pass these things 00:36:12.920 --> 00:36:15.103 align:middle line:84% around all the time called variance, 00:36:15.103 --> 00:36:16.520 align:middle line:84% and they're truly disgusting and I 00:36:16.520 --> 00:36:18.630 align:middle line:84% won't go into the details of what they are. 00:36:18.630 --> 00:36:21.380 align:middle line:84% But the point is you can't just set a variable 00:36:21.380 --> 00:36:22.550 align:middle line:90% and pass it around. 00:36:22.550 --> 00:36:24.110 align:middle line:84% In screen readerland, you're doing 00:36:24.110 --> 00:36:27.680 align:middle line:84% interprocess communication because your application is 00:36:27.680 --> 00:36:29.780 align:middle line:84% telling the operating system that something's 00:36:29.780 --> 00:36:31.940 align:middle line:84% available outside of your process, 00:36:31.940 --> 00:36:34.550 align:middle line:84% and then another application outside of your process, 00:36:34.550 --> 00:36:37.020 align:middle line:84% too, is retrieving it between there. 00:36:37.020 --> 00:36:38.780 align:middle line:84% So the only way to do that in Windows is 00:36:38.780 --> 00:36:42.350 align:middle line:84% do these funny variants and then send them around 00:36:42.350 --> 00:36:45.560 align:middle line:84% through this thing called a UIA raise automation 00:36:45.560 --> 00:36:48.290 align:middle line:90% property changed event. 00:36:48.290 --> 00:36:50.840 align:middle line:84% Effectively, what this is telling the operating system 00:36:50.840 --> 00:36:53.180 align:middle line:84% is that my checkbox has changed its value. 00:36:53.180 --> 00:36:56.520 align:middle line:84% My spinning, fire-breathing monkey is now on or off. 00:36:56.520 --> 00:36:58.880 align:middle line:84% And then a screen reader or other device 00:36:58.880 --> 00:37:00.450 align:middle line:84% can choose whether or not to tell 00:37:00.450 --> 00:37:02.610 align:middle line:90% the user about that effect. 00:37:02.610 --> 00:37:06.260 align:middle line:84% Last thing I'll mention, there's this funny line here 00:37:06.260 --> 00:37:08.330 align:middle line:84% that says if UIA clients are listening. 00:37:08.330 --> 00:37:10.550 align:middle line:84% In theory, what this function is supposed to do-- 00:37:10.550 --> 00:37:11.758 align:middle line:90% this is a Microsoft function. 00:37:11.758 --> 00:37:14.092 align:middle line:84% It's supposed to tell you if the screen reader is active 00:37:14.092 --> 00:37:16.220 align:middle line:84% and then respond differently if it's there or not. 00:37:16.220 --> 00:37:18.980 align:middle line:84% But actually, if you don't know, this doesn't actually work. 00:37:18.980 --> 00:37:21.830 align:middle line:84% It's just broken in Windows, as far as I know. 00:37:21.830 --> 00:37:23.990 align:middle line:84% I've confirmed this with Microsoft developers 00:37:23.990 --> 00:37:26.240 align:middle line:84% that as far as they can tell, it doesn't work, either. 00:37:26.240 --> 00:37:30.113 align:middle line:84% But as a convention, when you're writing accessibility code, 00:37:30.113 --> 00:37:32.030 align:middle line:84% you always want these little monikers in there 00:37:32.030 --> 00:37:34.400 align:middle line:84% just in case, because if they ever fix it, 00:37:34.400 --> 00:37:36.680 align:middle line:84% I'm scared as to what would happen if you 00:37:36.680 --> 00:37:38.330 align:middle line:90% don't have these in place. 00:37:38.330 --> 00:37:40.040 align:middle line:84% So unfortunately, this is the reality 00:37:40.040 --> 00:37:41.870 align:middle line:90% of accessibility coding. 00:37:41.870 --> 00:37:44.180 align:middle line:84% Sometimes you have to be defensive in ways 00:37:44.180 --> 00:37:47.570 align:middle line:84% that you wouldn't know make any sense at all, right? 00:37:47.570 --> 00:37:49.320 align:middle line:84% This is in the Microsoft documentation, 00:37:49.320 --> 00:37:50.960 align:middle line:84% but it doesn't work, yet you still 00:37:50.960 --> 00:37:53.030 align:middle line:90% have to do it because reasons. 00:37:53.030 --> 00:37:56.740 align:middle line:84% Welcome to accessibility technologies. 00:37:56.740 --> 00:37:57.640 align:middle line:90% OK. 00:37:57.640 --> 00:38:00.610 align:middle line:84% That's the idea of that particular type of thing. 00:38:00.610 --> 00:38:03.160 align:middle line:84% So now I'm going to switch back my shared 00:38:03.160 --> 00:38:09.640 align:middle line:84% screen to just the presentation and keep going here. 00:38:09.640 --> 00:38:12.280 align:middle line:90% 00:38:12.280 --> 00:38:16.560 align:middle line:84% OK, so in this particular case, the last-- 00:38:16.560 --> 00:38:19.400 align:middle line:90% uh, put this down. 00:38:19.400 --> 00:38:21.410 align:middle line:84% The last thing is for certain kinds of graphics, 00:38:21.410 --> 00:38:23.720 align:middle line:84% and this is where we get back to data science, 00:38:23.720 --> 00:38:27.140 align:middle line:84% graphics themselves aren't inherently accessible or not 00:38:27.140 --> 00:38:27.710 align:middle line:90% accessible. 00:38:27.710 --> 00:38:30.140 align:middle line:84% Any kind of graphics can be made accessible. 00:38:30.140 --> 00:38:32.990 align:middle line:84% But inside of operating systems, they 00:38:32.990 --> 00:38:34.748 align:middle line:84% don't always have providers for the kinds 00:38:34.748 --> 00:38:35.790 align:middle line:90% of things you would want. 00:38:35.790 --> 00:38:38.162 align:middle line:84% So for example, in the Quorum Studio product 00:38:38.162 --> 00:38:39.620 align:middle line:84% that we have that we use at schools 00:38:39.620 --> 00:38:42.770 align:middle line:84% for kids with disabilities, there 00:38:42.770 --> 00:38:45.110 align:middle line:84% is no such thing as an infinitely sized grid 00:38:45.110 --> 00:38:46.010 align:middle line:90% in a provider. 00:38:46.010 --> 00:38:46.880 align:middle line:90% That doesn't exist. 00:38:46.880 --> 00:38:49.490 align:middle line:84% So how do you do Unity 3D style things? 00:38:49.490 --> 00:38:50.990 align:middle line:84% It turns out you can, you just have 00:38:50.990 --> 00:38:53.640 align:middle line:84% to be kind of clever about how you manage your providers. 00:38:53.640 --> 00:38:56.600 align:middle line:84% There's also no control type for a scene designer. 00:38:56.600 --> 00:38:59.570 align:middle line:84% That's not a thing, Unity-style scene designers. 00:38:59.570 --> 00:39:01.640 align:middle line:90% But there are ways to do it. 00:39:01.640 --> 00:39:04.040 align:middle line:84% And I'm not going to go into the details about how, 00:39:04.040 --> 00:39:07.540 align:middle line:84% but making providers for content this graphical and accessible 00:39:07.540 --> 00:39:08.420 align:middle line:90% is totally possible. 00:39:08.420 --> 00:39:11.337 align:middle line:84% And we do it today and it works just fine. 00:39:11.337 --> 00:39:12.920 align:middle line:84% And it turns out if you solve problems 00:39:12.920 --> 00:39:15.500 align:middle line:84% like this, which we call the accessible graphics problem, 00:39:15.500 --> 00:39:20.060 align:middle line:84% you can do data science, block programming, 3D modeling, 00:39:20.060 --> 00:39:20.718 align:middle line:90% shaders. 00:39:20.718 --> 00:39:22.010 align:middle line:90% Doesn't matter if you're blind. 00:39:22.010 --> 00:39:23.250 align:middle line:90% Totally not a problem at all. 00:39:23.250 --> 00:39:24.900 align:middle line:90% It's just fine. 00:39:24.900 --> 00:39:28.030 align:middle line:84% And we solved that only recently, fully, in Quorum 8. 00:39:28.030 --> 00:39:29.680 align:middle line:84% So you can look at the implementations 00:39:29.680 --> 00:39:31.700 align:middle line:84% under [? BST, ?] so you can use it, as well. 00:39:31.700 --> 00:39:34.840 align:middle line:84% Point is solving the accessible graphics problem solves issues 00:39:34.840 --> 00:39:36.880 align:middle line:84% across a lot of fields, including data science, 00:39:36.880 --> 00:39:39.220 align:middle line:84% and you do it through these operating system hooks. 00:39:39.220 --> 00:39:40.720 align:middle line:84% That's the trick that you have to do 00:39:40.720 --> 00:39:43.460 align:middle line:90% to make all this stuff work. 00:39:43.460 --> 00:39:43.960 align:middle line:90% OK. 00:39:43.960 --> 00:39:47.380 align:middle line:84% So now let's move briefly to testing for accessibility. 00:39:47.380 --> 00:39:49.630 align:middle line:84% And then I'm going to show you some graphs that 00:39:49.630 --> 00:39:51.880 align:middle line:84% were designed through these successful graphics things 00:39:51.880 --> 00:39:53.713 align:middle line:84% so you can see their properties and see what 00:39:53.713 --> 00:39:58.480 align:middle line:84% they do in one particular AT device, which is a screen reader. 00:39:58.480 --> 00:40:00.850 align:middle line:84% So a couple of tools to be aware of when 00:40:00.850 --> 00:40:02.530 align:middle line:84% you're testing for accessibility, one 00:40:02.530 --> 00:40:03.700 align:middle line:90% is called Inspect. 00:40:03.700 --> 00:40:06.550 align:middle line:84% And I've put links to it in the slides. 00:40:06.550 --> 00:40:09.910 align:middle line:84% And another is called Accessibility Insights. 00:40:09.910 --> 00:40:12.430 align:middle line:84% If you're aware of it, Inspect is a little bit older. 00:40:12.430 --> 00:40:15.400 align:middle line:84% It looks really clunky, but it also works pretty well. 00:40:15.400 --> 00:40:17.950 align:middle line:84% Accessibility Insights exists and it's new 00:40:17.950 --> 00:40:20.350 align:middle line:84% and it looks really pretty, but as far as we can tell, 00:40:20.350 --> 00:40:22.810 align:middle line:84% we never really use it because it doesn't really 00:40:22.810 --> 00:40:24.970 align:middle line:84% work all the time and it's kind of funky. 00:40:24.970 --> 00:40:25.900 align:middle line:90% I don't know. 00:40:25.900 --> 00:40:27.370 align:middle line:84% It might just be us using it wrong, 00:40:27.370 --> 00:40:30.530 align:middle line:84% but we always use Inspect because of that reason. 00:40:30.530 --> 00:40:33.087 align:middle line:84% And these tools give you a lot of information 00:40:33.087 --> 00:40:33.920 align:middle line:90% about accessibility. 00:40:33.920 --> 00:40:36.110 align:middle line:84% And so I want to highlight on the screen, 00:40:36.110 --> 00:40:38.700 align:middle line:84% there's a little region where it says 00:40:38.700 --> 00:40:40.390 align:middle line:90% is annotation pattern available? 00:40:40.390 --> 00:40:41.650 align:middle line:90% IsDragPatternAvailable? 00:40:41.650 --> 00:40:43.390 align:middle line:90% Is drop target available? 00:40:43.390 --> 00:40:44.950 align:middle line:90% IsMultipleViewPatternAvailable? 00:40:44.950 --> 00:40:46.250 align:middle line:90% All these patterns. 00:40:46.250 --> 00:40:49.420 align:middle line:84% These are the literal patterns that come in from the code 00:40:49.420 --> 00:40:51.150 align:middle line:90% that I showed you earlier. 00:40:51.150 --> 00:40:53.262 align:middle line:84% And in addition, inside of this thing, 00:40:53.262 --> 00:40:55.220 align:middle line:84% there's also this thing that says control type. 00:40:55.220 --> 00:40:58.750 align:middle line:84% And it says UIA menu item control type ID. 00:40:58.750 --> 00:41:02.200 align:middle line:84% This is the literal thing provided by the provider 00:41:02.200 --> 00:41:05.800 align:middle line:84% to the operating system, and any accessibility technology 00:41:05.800 --> 00:41:08.470 align:middle line:90% can choose to read this or not. 00:41:08.470 --> 00:41:12.170 align:middle line:84% And the screen readers vary in what they choose to do. 00:41:12.170 --> 00:41:15.130 align:middle line:84% In other words, if you get your accessibility 00:41:15.130 --> 00:41:18.820 align:middle line:84% 100% right on the native operating system level, 00:41:18.820 --> 00:41:21.760 align:middle line:84% you still may not get support inside of a screen reader. 00:41:21.760 --> 00:41:24.970 align:middle line:84% And that's not necessarily wrong because different screen 00:41:24.970 --> 00:41:28.510 align:middle line:84% readers choose to interpret these events in different ways, 00:41:28.510 --> 00:41:30.200 align:middle line:84% and you kind of just have to know. 00:41:30.200 --> 00:41:33.070 align:middle line:84% As well, be aware, the screen reader JAWS 00:41:33.070 --> 00:41:36.820 align:middle line:84% is at least a few years behind on their support for UIA. 00:41:36.820 --> 00:41:39.250 align:middle line:84% And so as such, screen readers like JAWS 00:41:39.250 --> 00:41:41.080 align:middle line:84% actually give inconsistent information 00:41:41.080 --> 00:41:44.837 align:middle line:84% compared to screen readers like NVDA or even Windows Narrator, 00:41:44.837 --> 00:41:46.420 align:middle line:84% actually, is a little bit more correct 00:41:46.420 --> 00:41:48.560 align:middle line:84% on this particular front of interpretation. 00:41:48.560 --> 00:41:49.720 align:middle line:90% So just be aware. 00:41:49.720 --> 00:41:51.727 align:middle line:84% If you're getting trouble with one screen reader 00:41:51.727 --> 00:41:54.310 align:middle line:84% and it works in another, there's sometimes really good reasons 00:41:54.310 --> 00:41:57.010 align:middle line:84% for that, and you should ask an expert because sometimes they 00:41:57.010 --> 00:41:59.550 align:middle line:90% would know. 00:41:59.550 --> 00:42:00.050 align:middle line:90% OK. 00:42:00.050 --> 00:42:01.502 align:middle line:90% So let's take a quick demo. 00:42:01.502 --> 00:42:02.960 align:middle line:84% And I'm going to go back to sharing 00:42:02.960 --> 00:42:04.710 align:middle line:84% the screen up above so you can kind of see 00:42:04.710 --> 00:42:06.450 align:middle line:90% how this works in practice. 00:42:06.450 --> 00:42:09.500 align:middle line:84% And I'm going to pull up Quorum Studio, the latest 00:42:09.500 --> 00:42:11.870 align:middle line:90% version, which is 2.2. 00:42:11.870 --> 00:42:15.170 align:middle line:84% And inside of it, I'm also going to pull up the Inspect 00:42:15.170 --> 00:42:17.090 align:middle line:84% tool first, without a screen reader active, 00:42:17.090 --> 00:42:19.810 align:middle line:84% so you can see what kind of information is provided. 00:42:19.810 --> 00:42:24.560 align:middle line:84% And then I'll show you some stuff with some graphs. 00:42:24.560 --> 00:42:25.060 align:middle line:90% OK. 00:42:25.060 --> 00:42:27.130 align:middle line:84% So hopefully you can see my screen again. 00:42:27.130 --> 00:42:30.070 align:middle line:84% And I'm going to pull up this tool, inspect. 00:42:30.070 --> 00:42:31.930 align:middle line:84% And I'm going to just click quorum studio 00:42:31.930 --> 00:42:34.760 align:middle line:84% and I want you to see what's in there. 00:42:34.760 --> 00:42:37.630 align:middle line:84% So inside of this, you'll notice that visually on the screen, 00:42:37.630 --> 00:42:39.520 align:middle line:84% believe it or not, Quorum studio is actually 00:42:39.520 --> 00:42:41.470 align:middle line:90% not native controls. 00:42:41.470 --> 00:42:46.420 align:middle line:84% It looks like it has tabs and it looks like it has little icons 00:42:46.420 --> 00:42:49.378 align:middle line:84% and it looks like it has trees and affordances and all 00:42:49.378 --> 00:42:50.170 align:middle line:90% this kind of stuff. 00:42:50.170 --> 00:42:51.070 align:middle line:90% It's not. 00:42:51.070 --> 00:42:54.130 align:middle line:84% It's raw OpenGL that is rendered on the graphics chip. 00:42:54.130 --> 00:42:56.860 align:middle line:84% It has nothing to do with native controls whatsoever. 00:42:56.860 --> 00:42:59.890 align:middle line:84% It's a big total lie to the operating 00:42:59.890 --> 00:43:01.810 align:middle line:90% system as to what is happening. 00:43:01.810 --> 00:43:02.990 align:middle line:90% Now why is that? 00:43:02.990 --> 00:43:05.890 align:middle line:84% That's because we want to allow 3D things to be accessible. 00:43:05.890 --> 00:43:07.480 align:middle line:84% We want graphics to be accessible. 00:43:07.480 --> 00:43:10.105 align:middle line:84% We want graphs to be accessible and we really can't with native 00:43:10.105 --> 00:43:11.990 align:middle line:84% controls because they don't work that way. 00:43:11.990 --> 00:43:14.080 align:middle line:84% So if you look inside of this, you'll 00:43:14.080 --> 00:43:15.670 align:middle line:84% notice that what we've done then is 00:43:15.670 --> 00:43:17.380 align:middle line:90% we've set a bunch of providers. 00:43:17.380 --> 00:43:20.490 align:middle line:84% So right now, it looks like it has the projects tree, 00:43:20.490 --> 00:43:23.200 align:middle line:84% and that is a UIA tree control type ID. 00:43:23.200 --> 00:43:25.120 align:middle line:84% So a screen reader or other AT device 00:43:25.120 --> 00:43:26.620 align:middle line:84% would think that that's just a tree. 00:43:26.620 --> 00:43:28.510 align:middle line:84% They wouldn't know that there's anything different. 00:43:28.510 --> 00:43:29.560 align:middle line:90% It's just a normal tree. 00:43:29.560 --> 00:43:31.950 align:middle line:90% I can use it in the normal way. 00:43:31.950 --> 00:43:34.300 align:middle line:90% Same with if I have a text box. 00:43:34.300 --> 00:43:38.030 align:middle line:84% In this particular case, that's a UIA document control type ID. 00:43:38.030 --> 00:43:40.020 align:middle line:84% And you can notice that when I click the focus 00:43:40.020 --> 00:43:42.540 align:middle line:84% inside of Quorum studio, the operating system 00:43:42.540 --> 00:43:47.130 align:middle line:84% is informed that this particular type of thing has changed. 00:43:47.130 --> 00:43:49.800 align:middle line:84% And then inside of the providers, if you don't know, 00:43:49.800 --> 00:43:52.560 align:middle line:84% implementing a text provider's easily the hardest one. 00:43:52.560 --> 00:43:55.470 align:middle line:84% It's very, very difficult to get it exactly right. 00:43:55.470 --> 00:43:59.970 align:middle line:84% By having the text pattern available and the volume 00:43:59.970 --> 00:44:01.440 align:middle line:84% pattern available, all of a sudden, 00:44:01.440 --> 00:44:03.532 align:middle line:84% you get Braille support for free. 00:44:03.532 --> 00:44:05.490 align:middle line:84% And you wouldn't even know that unless you just 00:44:05.490 --> 00:44:08.770 align:middle line:84% magically know that this is how this works internally. 00:44:08.770 --> 00:44:11.460 align:middle line:84% And so this is one reason why a tool like Quorum studio 00:44:11.460 --> 00:44:13.800 align:middle line:84% can connect to Braille devices, because we just 00:44:13.800 --> 00:44:15.930 align:middle line:84% know that these text patterns are related 00:44:15.930 --> 00:44:18.180 align:middle line:84% to the way Braille support is done because the Braille 00:44:18.180 --> 00:44:21.090 align:middle line:84% support connects to UIA just like a screen reader would 00:44:21.090 --> 00:44:23.900 align:middle line:90% or any other device. 00:44:23.900 --> 00:44:27.240 align:middle line:84% OK, so let's pull up a chart inside of this. 00:44:27.240 --> 00:44:29.290 align:middle line:84% So I'm going to pull up a data science library. 00:44:29.290 --> 00:44:31.420 align:middle line:84% I've been working on a statistics library, a data 00:44:31.420 --> 00:44:33.337 align:middle line:84% science library, for, specifically, the Quorum 00:44:33.337 --> 00:44:34.140 align:middle line:90% language. 00:44:34.140 --> 00:44:37.090 align:middle line:84% And if I pull this up and I look at the UIA, 00:44:37.090 --> 00:44:39.460 align:middle line:90% I have on screen, a small-- 00:44:39.460 --> 00:44:41.320 align:middle line:90% I mean, I just did it that way. 00:44:41.320 --> 00:44:43.220 align:middle line:90% A small little chart. 00:44:43.220 --> 00:44:48.240 align:middle line:84% And when I pull this up and that gets the focus, 00:44:48.240 --> 00:44:50.490 align:middle line:84% you can notice that in UIA, there's actually 00:44:50.490 --> 00:44:53.730 align:middle line:90% a lot of junk that gets put in. 00:44:53.730 --> 00:44:56.055 align:middle line:84% And this looks like it's giving a lot of information. 00:44:56.055 --> 00:44:59.160 align:middle line:84% It's putting in individual points and all sorts of stuff. 00:44:59.160 --> 00:45:01.462 align:middle line:84% The structure of this tree, because of the way 00:45:01.462 --> 00:45:03.420 align:middle line:84% we're doing it internally for events like this, 00:45:03.420 --> 00:45:04.180 align:middle line:90% is kind of funny. 00:45:04.180 --> 00:45:06.930 align:middle line:84% So just kind of ignore the fact that it doesn't look tree-like. 00:45:06.930 --> 00:45:08.420 align:middle line:84% But from a user's perspective, it 00:45:08.420 --> 00:45:11.640 align:middle line:84% doesn't matter at all, because we've decided, we've defined, 00:45:11.640 --> 00:45:13.840 align:middle line:84% how they interact with these charts. 00:45:13.840 --> 00:45:16.230 align:middle line:84% And in addition, you'll notice that inside of here, 00:45:16.230 --> 00:45:20.190 align:middle line:84% there's this funny thing called a UIA custom control type ID. 00:45:20.190 --> 00:45:23.500 align:middle line:84% This is a flag to the operating system that says, hey, 00:45:23.500 --> 00:45:25.110 align:middle line:90% we know what we're doing. 00:45:25.110 --> 00:45:27.000 align:middle line:90% We will take care of it for you. 00:45:27.000 --> 00:45:28.860 align:middle line:90% Just chill. 00:45:28.860 --> 00:45:31.380 align:middle line:84% And so when you do that, you have 00:45:31.380 --> 00:45:34.020 align:middle line:84% to be very careful about what you're telling the operating 00:45:34.020 --> 00:45:37.620 align:middle line:84% system at what times because you don't get things like selection 00:45:37.620 --> 00:45:40.320 align:middle line:84% like you would in a tree, which are hooks to specifically 00:45:40.320 --> 00:45:40.950 align:middle line:90% screen readers. 00:45:40.950 --> 00:45:43.410 align:middle line:84% Instead, you have to manipulate the system 00:45:43.410 --> 00:45:46.050 align:middle line:84% through focus events, not through selection events, which 00:45:46.050 --> 00:45:48.520 align:middle line:84% is kind of an esoteric detail, but it's important. 00:45:48.520 --> 00:45:52.320 align:middle line:84% So when I do this discreetly, you'll see what I mean. 00:45:52.320 --> 00:45:55.160 align:middle line:84% So I'm going to pop this up NVDA. 00:45:55.160 --> 00:45:56.500 align:middle line:90% NVDA: [INAUDIBLE] 00:45:56.500 --> 00:46:01.390 align:middle line:90% 00:46:01.390 --> 00:46:03.460 align:middle line:84% ANDREAS STEFIK: And I apologize that the scaling 00:46:03.460 --> 00:46:04.710 align:middle line:90% is a little bit funny on here. 00:46:04.710 --> 00:46:06.550 align:middle line:84% We're having a couple issues with scaling. 00:46:06.550 --> 00:46:08.000 align:middle line:84% This is all brand new technology. 00:46:08.000 --> 00:46:09.375 align:middle line:84% We literally deployed it Tuesday, 00:46:09.375 --> 00:46:12.640 align:middle line:84% so it's not like some long-term thing 00:46:12.640 --> 00:46:14.600 align:middle line:90% that we've had for a long time. 00:46:14.600 --> 00:46:16.810 align:middle line:84% You'll notice that if I have a screen reader on 00:46:16.810 --> 00:46:19.810 align:middle line:84% and I press the arrow keys, this looks like a chart, 00:46:19.810 --> 00:46:21.370 align:middle line:90% really, to a sighted user. 00:46:21.370 --> 00:46:23.260 align:middle line:84% You wouldn't know that this is not a chart. 00:46:23.260 --> 00:46:25.660 align:middle line:84% It just displays graphics like anything else. 00:46:25.660 --> 00:46:27.515 align:middle line:90% But If I press the arrow keys-- 00:46:27.515 --> 00:46:31.555 align:middle line:84% NVDA: [INAUDIBLE] from 0.0 to 1,120.0. 00:46:31.555 --> 00:46:33.430 align:middle line:84% ANDREAS STEFIK: I can interact with the chart 00:46:33.430 --> 00:46:34.960 align:middle line:90% in any kind of meaningful way. 00:46:34.960 --> 00:46:37.480 align:middle line:84% This is exactly like what Jonathan Godfrey put 00:46:37.480 --> 00:46:40.210 align:middle line:84% in one of his publications, except instead 00:46:40.210 --> 00:46:42.070 align:middle line:84% of just providing structure randomly, 00:46:42.070 --> 00:46:45.490 align:middle line:84% it provides operating system hooks so that the user would 00:46:45.490 --> 00:46:48.370 align:middle line:84% just not know that this just magically is 00:46:48.370 --> 00:46:49.510 align:middle line:90% accessible out of the box. 00:46:49.510 --> 00:46:51.530 align:middle line:84% They don't even have to do any extra work. 00:46:51.530 --> 00:46:53.780 align:middle line:84% In addition to that, if I press the up and down keys-- 00:46:53.780 --> 00:46:55.020 align:middle line:90% NVDA: [INAUDIBLE] 00:46:55.020 --> 00:46:56.812 align:middle line:84% ANDREAS STEFIK: I can interact with things. 00:46:56.812 --> 00:46:57.990 align:middle line:90% NVDA: 1,120.0505. 00:46:57.990 --> 00:46:59.740 align:middle line:84% ANDREAS STEFIK: I can interact with pieces 00:46:59.740 --> 00:47:01.840 align:middle line:84% of the chart in ways that we thought 00:47:01.840 --> 00:47:03.850 align:middle line:90% made sense for accessibility. 00:47:03.850 --> 00:47:07.390 align:middle line:84% Now of course, we haven't done user vetting with users yet 00:47:07.390 --> 00:47:09.880 align:middle line:84% to know exactly which pieces people will like and which 00:47:09.880 --> 00:47:11.060 align:middle line:90% pieces they won't. 00:47:11.060 --> 00:47:13.630 align:middle line:84% But the point is that by having this UIA structure 00:47:13.630 --> 00:47:16.990 align:middle line:84% and defining a set of keys, now we can go do testing with users 00:47:16.990 --> 00:47:20.060 align:middle line:84% and see which things they want changed and stuff like that. 00:47:20.060 --> 00:47:21.580 align:middle line:84% So if I get to the actual plots-- 00:47:21.580 --> 00:47:23.290 align:middle line:90% NVDA: FS scale is [INAUDIBLE]. 00:47:23.290 --> 00:47:24.726 align:middle line:90% Scatter plot 25 points. 00:47:24.726 --> 00:47:27.910 align:middle line:90% VP 0.0 to 11,460.0. 00:47:27.910 --> 00:47:31.134 align:middle line:90% [INAUDIBLE] to 1,120.0. 00:47:31.134 --> 00:47:32.635 align:middle line:90% [INAUDIBLE] 00:47:32.635 --> 00:47:34.510 align:middle line:84% ANDREAS STEFIK: This gets a little bit tricky 00:47:34.510 --> 00:47:36.760 align:middle line:84% because if you had something like a scatter plot, 00:47:36.760 --> 00:47:39.523 align:middle line:84% hypothetically, this could be thousands or millions of points 00:47:39.523 --> 00:47:40.940 align:middle line:84% that you might have on the screen. 00:47:40.940 --> 00:47:42.988 align:middle line:84% So the question is, how do you deal with that? 00:47:42.988 --> 00:47:44.030 align:middle line:90% There's a number of ways. 00:47:44.030 --> 00:47:46.990 align:middle line:84% One is, I think I saw Sean Mealin on the call earlier. 00:47:46.990 --> 00:47:48.723 align:middle line:90% NVDA: [INAUDIBLE] 00:47:48.723 --> 00:47:50.140 align:middle line:84% ANDREAS STEFIK: But in any case, I 00:47:50.140 --> 00:47:52.330 align:middle line:84% know with stuff like the graphics accelerator, 00:47:52.330 --> 00:47:54.820 align:middle line:84% they try to give you the gestalt through sonification. 00:47:54.820 --> 00:47:56.790 align:middle line:84% That's one way that you can try to do it. 00:47:56.790 --> 00:47:59.630 align:middle line:84% You might also want to get at the individual points. 00:47:59.630 --> 00:48:02.710 align:middle line:84% So one of the ideas we had here was 00:48:02.710 --> 00:48:04.600 align:middle line:84% we can actually pretend that a scatter 00:48:04.600 --> 00:48:06.640 align:middle line:90% plot is, in fact, a quad tree. 00:48:06.640 --> 00:48:08.730 align:middle line:84% If you're not a graphics person, this 00:48:08.730 --> 00:48:11.230 align:middle line:84% is just a way of breaking up the structure in a certain way. 00:48:11.230 --> 00:48:13.130 align:middle line:84% And it looks kind of like this on screen. 00:48:13.130 --> 00:48:13.960 align:middle line:90% NVDA: [INAUDIBLE] 00:48:13.960 --> 00:48:14.230 align:middle line:90% ANDREAS STEFIK: Oops. 00:48:14.230 --> 00:48:15.963 align:middle line:90% Does not look like that at all. 00:48:15.963 --> 00:48:16.850 align:middle line:90% NVDA: FS scale. 00:48:16.850 --> 00:48:20.270 align:middle line:84% Scatter plot 20 [INAUDIBLE] quadrant [INAUDIBLE] 00:48:20.270 --> 00:48:23.110 align:middle line:90% to 5,730.0. 00:48:23.110 --> 00:48:26.110 align:middle line:90% FS 560.0 to 1,120.0. 00:48:26.110 --> 00:48:28.360 align:middle line:84% ANDREAS STEFIK: I can break up my thing into quadrants 00:48:28.360 --> 00:48:30.350 align:middle line:84% and give it information about what is in there. 00:48:30.350 --> 00:48:31.850 align:middle line:84% In this particular case, it's saying 00:48:31.850 --> 00:48:33.470 align:middle line:84% there's no points in this region, 00:48:33.470 --> 00:48:35.720 align:middle line:84% here's the scale for this particular region, 00:48:35.720 --> 00:48:37.540 align:middle line:84% so you kind of know where you're at. 00:48:37.540 --> 00:48:39.903 align:middle line:84% My hunch is when we do the user testing, 00:48:39.903 --> 00:48:42.070 align:middle line:84% they'll want us to shorten these queues a little bit 00:48:42.070 --> 00:48:44.260 align:middle line:84% to make them a little easier to understand, 00:48:44.260 --> 00:48:46.537 align:middle line:84% and they'll probably want us to change, maybe remove 00:48:46.537 --> 00:48:48.370 align:middle line:84% some names, or do some adjustment like that. 00:48:48.370 --> 00:48:51.342 align:middle line:84% But we'll find that out when we start doing testing. 00:48:51.342 --> 00:48:52.800 align:middle line:84% NVDA: Top right quadrant 11 points. 00:48:52.800 --> 00:48:57.460 align:middle line:90% VP 5,730.0 to 11,460.0. 00:48:57.460 --> 00:49:01.350 align:middle line:90% FS 560.0 to 1,120.0. 00:49:01.350 --> 00:49:03.190 align:middle line:84% [INAUDIBLE] to inspect the top region. 00:49:03.190 --> 00:49:06.290 align:middle line:84% ANDREAS STEFIK: Now in quad trees, the brilliance of them 00:49:06.290 --> 00:49:06.860 align:middle line:90% is that-- 00:49:06.860 --> 00:49:09.360 align:middle line:84% I mean, at least, the thing that I think is cool about them, 00:49:09.360 --> 00:49:11.010 align:middle line:84% is that even when you get to a region, 00:49:11.010 --> 00:49:12.427 align:middle line:84% you might know this has 11 points. 00:49:12.427 --> 00:49:15.098 align:middle line:84% You might want to drill down a little further, so you can. 00:49:15.098 --> 00:49:17.390 align:middle line:84% NVDA: Top left subregion-- top right sub-- bottom right 00:49:17.390 --> 00:49:18.642 align:middle line:90% subregion 6 points. 00:49:18.642 --> 00:49:22.457 align:middle line:90% VP 8,595.0 to 11,460.0. 00:49:22.457 --> 00:49:23.790 align:middle line:90% ANDREAS STEFIK: So I'll stop it. 00:49:23.790 --> 00:49:25.165 align:middle line:84% But the idea is that you can then 00:49:25.165 --> 00:49:27.118 align:middle line:84% drill down to individual parts, eventually 00:49:27.118 --> 00:49:28.160 align:middle line:90% to the points themselves. 00:49:28.160 --> 00:49:29.900 align:middle line:90% NVDA: VP 9,025.0. 00:49:29.900 --> 00:49:31.910 align:middle line:90% FS 680.0. 00:49:31.910 --> 00:49:32.610 align:middle line:90% [INAUDIBLE] 00:49:32.610 --> 00:49:34.443 align:middle line:84% ANDREAS STEFIK: And it gives you information 00:49:34.443 --> 00:49:36.980 align:middle line:84% like what is this individual point, what 00:49:36.980 --> 00:49:39.390 align:middle line:84% is the ordinality of this point in this particular region 00:49:39.390 --> 00:49:40.223 align:middle line:90% and the cardinality? 00:49:40.223 --> 00:49:42.740 align:middle line:84% So, like, one of six, how many points are in this region? 00:49:42.740 --> 00:49:44.330 align:middle line:84% And you can traverse through them. 00:49:44.330 --> 00:49:46.672 align:middle line:90% NVDA: [INAUDIBLE] 00:49:46.672 --> 00:49:47.673 align:middle line:90% 00:49:47.673 --> 00:49:50.090 align:middle line:84% ANDREAS STEFIK: And I won't go through every little detail 00:49:50.090 --> 00:49:52.370 align:middle line:84% of the queues, but that's the idea. 00:49:52.370 --> 00:49:57.560 align:middle line:84% So overall, the idea with making something like data science 00:49:57.560 --> 00:49:59.390 align:middle line:84% more accessible to people with disabilities 00:49:59.390 --> 00:50:02.390 align:middle line:84% is you need a mapping to the operating system 00:50:02.390 --> 00:50:05.732 align:middle line:84% and you need to somehow provide structure for all of the items 00:50:05.732 --> 00:50:07.940 align:middle line:84% that are in there, in addition to the other things we 00:50:07.940 --> 00:50:10.550 align:middle line:84% mentioned with text and making the language easier to use 00:50:10.550 --> 00:50:12.240 align:middle line:90% and stuff like that. 00:50:12.240 --> 00:50:13.728 align:middle line:90% So I hope that-- 00:50:13.728 --> 00:50:14.270 align:middle line:90% I don't know. 00:50:14.270 --> 00:50:16.432 align:middle line:90% NVDA: [INAUDIBLE] 00:50:16.432 --> 00:50:16.908 align:middle line:90% 00:50:16.908 --> 00:50:19.200 align:middle line:84% ANDREAS STEFIK: I hope that this was interesting to you 00:50:19.200 --> 00:50:21.990 align:middle line:84% and that you had some information that 00:50:21.990 --> 00:50:23.430 align:middle line:84% might be useful to you when trying 00:50:23.430 --> 00:50:25.950 align:middle line:84% to think about data science or to do it accessibly. 00:50:25.950 --> 00:50:29.520 align:middle line:84% And at this point, I will open up for questions. 00:50:29.520 --> 00:50:32.527 align:middle line:90% So thanks for coming. 00:50:32.527 --> 00:50:34.860 align:middle line:84% BRIANNA BLASER: Folks, you can ask questions in the chat 00:50:34.860 --> 00:50:38.810 align:middle line:84% or unmute yourself and ask them verbally. 00:50:38.810 --> 00:50:40.525 align:middle line:84% STUDENT 1: Stefik, this is Richard. 00:50:40.525 --> 00:50:41.400 align:middle line:90% ANDREAS STEFIK: Yeah? 00:50:41.400 --> 00:50:42.733 align:middle line:90% STUDENT 1: Yeah, that was great. 00:50:42.733 --> 00:50:45.550 align:middle line:90% I turned my video off. 00:50:45.550 --> 00:50:47.630 align:middle line:84% I have a question about the distinction 00:50:47.630 --> 00:50:51.100 align:middle line:90% between a Mac and the iOS. 00:50:51.100 --> 00:50:55.220 align:middle line:84% Are there different accessibility approaches 00:50:55.220 --> 00:50:57.085 align:middle line:90% to those two platforms? 00:50:57.085 --> 00:50:57.960 align:middle line:90% ANDREAS STEFIK: Yeah. 00:50:57.960 --> 00:51:00.890 align:middle line:84% So they use different architectures. 00:51:00.890 --> 00:51:02.120 align:middle line:90% On windows, it's UIA. 00:51:02.120 --> 00:51:04.730 align:middle line:84% On Mac, it's called NSAccessibility. 00:51:04.730 --> 00:51:08.150 align:middle line:84% And on the one hand, they have a lot 00:51:08.150 --> 00:51:10.008 align:middle line:90% of philosophical similarities. 00:51:10.008 --> 00:51:11.300 align:middle line:90% You still do events and stuff-- 00:51:11.300 --> 00:51:14.105 align:middle line:84% STUDENT 1: I asked about iOS, not Mac. 00:51:14.105 --> 00:51:15.230 align:middle line:90% ANDREAS STEFIK: Oh, on iOS. 00:51:15.230 --> 00:51:17.360 align:middle line:84% I think they also use NSAccessibility. 00:51:17.360 --> 00:51:19.562 align:middle line:84% But because of the touch system, I 00:51:19.562 --> 00:51:21.770 align:middle line:84% think the events are different, if I recall correctly 00:51:21.770 --> 00:51:23.430 align:middle line:90% off the top my head, so. 00:51:23.430 --> 00:51:25.520 align:middle line:84% But I think if you add support for one, 00:51:25.520 --> 00:51:29.570 align:middle line:84% you might not get the other for free, but you get a lot of it, 00:51:29.570 --> 00:51:32.050 align:middle line:90% if I recall correctly, so. 00:51:32.050 --> 00:51:33.300 align:middle line:90% STUDENT 2: Yeah, that's right. 00:51:33.300 --> 00:51:34.590 align:middle line:90% They're very similar. 00:51:34.590 --> 00:51:36.132 align:middle line:84% ANDREAS STEFIK: They're very similar. 00:51:36.132 --> 00:51:37.220 align:middle line:90% Yeah. 00:51:37.220 --> 00:51:38.965 align:middle line:90% I see a couple questions. 00:51:38.965 --> 00:51:40.340 align:middle line:84% How difficult would it be to make 00:51:40.340 --> 00:51:41.910 align:middle line:90% web graphs that are accessible? 00:51:41.910 --> 00:51:43.470 align:middle line:90% Yeah, that's a good question. 00:51:43.470 --> 00:51:48.080 align:middle line:84% So it's possible, I think, using stuff 00:51:48.080 --> 00:51:49.670 align:middle line:84% like Shadow DOM and things like that 00:51:49.670 --> 00:51:52.140 align:middle line:90% to make that more available. 00:51:52.140 --> 00:51:54.420 align:middle line:90% But it is tricky. 00:51:54.420 --> 00:51:56.858 align:middle line:84% It's one of the things we've wanted to do on our projects 00:51:56.858 --> 00:51:59.150 align:middle line:84% for a while, is to add some support for stuff like that 00:51:59.150 --> 00:52:00.740 align:middle line:84% into our accessible graphing thing. 00:52:00.740 --> 00:52:02.115 align:middle line:90% But it's a little complicated. 00:52:02.115 --> 00:52:03.740 align:middle line:84% You still have to add structure and you 00:52:03.740 --> 00:52:04.940 align:middle line:90% have to make it traversable. 00:52:04.940 --> 00:52:05.982 align:middle line:90% So that's really the key. 00:52:05.982 --> 00:52:08.440 align:middle line:84% How do you end up doing that so that the screen readers can 00:52:08.440 --> 00:52:09.020 align:middle line:90% hook in? 00:52:09.020 --> 00:52:11.360 align:middle line:84% I've experimented some with Shadow DOM-style 00:52:11.360 --> 00:52:13.775 align:middle line:84% implementations for this internally on the Quorum 00:52:13.775 --> 00:52:15.650 align:middle line:84% project because we have a JavaScript version, 00:52:15.650 --> 00:52:18.770 align:middle line:84% but we don't have anything deployed for that now. 00:52:18.770 --> 00:52:22.700 align:middle line:84% So it's possible, but the web is tricky 00:52:22.700 --> 00:52:25.730 align:middle line:84% because the accessibility is not quite as mature 00:52:25.730 --> 00:52:28.758 align:middle line:84% as desktop operating systems are. 00:52:28.758 --> 00:52:29.300 align:middle line:90% I don't know. 00:52:29.300 --> 00:52:32.210 align:middle line:84% It works really well for navigating headers and stuff, 00:52:32.210 --> 00:52:36.110 align:middle line:84% but for complicated graphics, it is a little funky, I guess. 00:52:36.110 --> 00:52:37.822 align:middle line:84% I see Andrew shaking his head, so I'm 00:52:37.822 --> 00:52:39.780 align:middle line:84% guessing I'm not the only one that thinks that. 00:52:39.780 --> 00:52:42.390 align:middle line:90% But it's a little funny, so. 00:52:42.390 --> 00:52:46.322 align:middle line:84% STUDENT 1: Stefik, some people have their hand up in the-- 00:52:46.322 --> 00:52:48.530 align:middle line:84% BRIANNA BLASER: Yeah, I see JooYoung has his hand up. 00:52:48.530 --> 00:52:49.190 align:middle line:90% ANDREAS STEFIK: Oh, they do. 00:52:49.190 --> 00:52:49.730 align:middle line:90% Yeah. 00:52:49.730 --> 00:52:51.575 align:middle line:90% Please. 00:52:51.575 --> 00:52:52.950 align:middle line:84% JOOYOUNG: Before everything else, 00:52:52.950 --> 00:52:56.420 align:middle line:84% thank you very much for your awesome presentation, Doctor 00:52:56.420 --> 00:52:56.990 align:middle line:90% Stefik. 00:52:56.990 --> 00:52:57.782 align:middle line:90% That's really cool. 00:52:57.782 --> 00:52:58.907 align:middle line:90% ANDREAS STEFIK: No problem. 00:52:58.907 --> 00:52:59.914 align:middle line:90% And thanks for coming. 00:52:59.914 --> 00:53:01.770 align:middle line:90% JOOYOUNG: Yeah. 00:53:01.770 --> 00:53:04.100 align:middle line:84% I was wondering-- before everything else, 00:53:04.100 --> 00:53:11.980 align:middle line:84% I, as a actual user of SAS graphic accelerator, 00:53:11.980 --> 00:53:15.740 align:middle line:84% I think there is a way to navigate 00:53:15.740 --> 00:53:18.200 align:middle line:84% scatter plot by cell by cell, meaning 00:53:18.200 --> 00:53:23.570 align:middle line:90% that as quadratic layer. 00:53:23.570 --> 00:53:27.870 align:middle line:84% So not only you can navigate from left to right or top 00:53:27.870 --> 00:53:33.980 align:middle line:84% to bottom, et cetera, but you can navigate cell by cell. 00:53:33.980 --> 00:53:39.570 align:middle line:84% So I think that's a very similar concept that what you 00:53:39.570 --> 00:53:43.045 align:middle line:90% mean by structure navigation? 00:53:43.045 --> 00:53:44.010 align:middle line:90% ANDREAS STEFIK: Mm-hmm. 00:53:44.010 --> 00:53:45.468 align:middle line:84% JOOYOUNG: So I was wondering if you 00:53:45.468 --> 00:53:51.480 align:middle line:84% are considering sonification on top of that structure. 00:53:51.480 --> 00:53:59.280 align:middle line:84% Because right now, I guess, it would be really great 00:53:59.280 --> 00:54:04.830 align:middle line:84% if we have spatial information added using sonification, 00:54:04.830 --> 00:54:09.750 align:middle line:84% as well, at least a toggle switch. 00:54:09.750 --> 00:54:11.167 align:middle line:84% What do you think about that idea? 00:54:11.167 --> 00:54:12.917 align:middle line:84% ANDREAS STEFIK: I think it's a great idea. 00:54:12.917 --> 00:54:15.120 align:middle line:84% I mean, there's quite a bit of research 00:54:15.120 --> 00:54:18.660 align:middle line:84% on the use of sonification, like out of the International 00:54:18.660 --> 00:54:20.130 align:middle line:90% Conference on Auditory Display. 00:54:20.130 --> 00:54:22.620 align:middle line:84% And there's some other stuff, as well. 00:54:22.620 --> 00:54:25.030 align:middle line:84% And on the graphics accelerator, as well, 00:54:25.030 --> 00:54:27.930 align:middle line:84% I think they've done a really good job of trying to give you 00:54:27.930 --> 00:54:30.578 align:middle line:90% the Gestalt feel of a graph. 00:54:30.578 --> 00:54:32.620 align:middle line:84% And, I mean, their team might disagree with that. 00:54:32.620 --> 00:54:33.162 align:middle line:90% I don't know. 00:54:33.162 --> 00:54:36.270 align:middle line:84% But that's my impression, is that sort of stuff 00:54:36.270 --> 00:54:39.090 align:middle line:84% is what they've kind of tried to give you. 00:54:39.090 --> 00:54:40.600 align:middle line:90% I think it's good. 00:54:40.600 --> 00:54:42.690 align:middle line:84% I'm not an expert on the literature, 00:54:42.690 --> 00:54:46.050 align:middle line:84% but I do suspect that if we added stuff 00:54:46.050 --> 00:54:49.050 align:middle line:84% like that in for data science, it would be useful. 00:54:49.050 --> 00:54:51.960 align:middle line:84% I don't know exactly what context and when it would be 00:54:51.960 --> 00:54:53.070 align:middle line:90% and when it wouldn't. 00:54:53.070 --> 00:54:54.610 align:middle line:90% I'll give you an example. 00:54:54.610 --> 00:54:58.680 align:middle line:84% One of the teams I work with does robotic telescopes 00:54:58.680 --> 00:55:00.772 align:middle line:90% for young, blind people. 00:55:00.772 --> 00:55:02.730 align:middle line:84% And so they have these robotic telescope arrays 00:55:02.730 --> 00:55:04.890 align:middle line:84% and they get back these astronomical images. 00:55:04.890 --> 00:55:07.390 align:middle line:84% And they've been experimenting with sonification there. 00:55:07.390 --> 00:55:10.060 align:middle line:84% So for stuff like that, because it's not a scatter plot 00:55:10.060 --> 00:55:12.520 align:middle line:84% where it might just go up, it's a lot harder 00:55:12.520 --> 00:55:15.240 align:middle line:84% to figure out what the sonification tells you, right? 00:55:15.240 --> 00:55:20.730 align:middle line:84% Are you sonifying globular clusters or spiral galaxies? 00:55:20.730 --> 00:55:22.530 align:middle line:84% Does it sound different for each? 00:55:22.530 --> 00:55:25.290 align:middle line:84% So I don't think it's an all or one. 00:55:25.290 --> 00:55:26.910 align:middle line:84% I imagine sonification would have 00:55:26.910 --> 00:55:29.220 align:middle line:84% to be used on kind of a case-by-case basis, 00:55:29.220 --> 00:55:31.560 align:middle line:84% depending upon what you're trying to sonify. 00:55:31.560 --> 00:55:33.022 align:middle line:90% But that's just my guess. 00:55:33.022 --> 00:55:34.980 align:middle line:84% I don't really have strong evidence either way, 00:55:34.980 --> 00:55:39.800 align:middle line:84% I just think it's cool and would probably work for some things. 00:55:39.800 --> 00:55:43.315 align:middle line:84% JOOYOUNG: Can I kind of have piggyback question? 00:55:43.315 --> 00:55:44.190 align:middle line:90% ANDREAS STEFIK: Sure. 00:55:44.190 --> 00:55:44.815 align:middle line:90% JOOYOUNG: Yeah. 00:55:44.815 --> 00:55:52.550 align:middle line:84% So I think auto-embedding accessibility 00:55:52.550 --> 00:55:59.600 align:middle line:84% API, like you did, is really cool and necessary. 00:55:59.600 --> 00:56:05.760 align:middle line:84% I think someone asked about web graphic, as well. 00:56:05.760 --> 00:56:11.780 align:middle line:84% In my understanding, Jonathan Godfrey and their team, 00:56:11.780 --> 00:56:18.390 align:middle line:84% they are aiming at SVG as a vehicle 00:56:18.390 --> 00:56:22.900 align:middle line:84% to deliver like MathML type, zoom in and zoom out 00:56:22.900 --> 00:56:24.930 align:middle line:90% navigation. 00:56:24.930 --> 00:56:26.520 align:middle line:90% Actually, I have the experience. 00:56:26.520 --> 00:56:33.780 align:middle line:84% There are some prototypes for histogram and box plot. 00:56:33.780 --> 00:56:40.200 align:middle line:84% I think that would be great if someone 00:56:40.200 --> 00:56:46.380 align:middle line:84% can develop some SVG plus auto zoom in, 00:56:46.380 --> 00:56:51.480 align:middle line:90% zoom out, just like MathML. 00:56:51.480 --> 00:56:55.050 align:middle line:84% So would there be any possibility for your team 00:56:55.050 --> 00:56:59.647 align:middle line:90% to develop that support? 00:56:59.647 --> 00:57:01.480 align:middle line:84% ANDREAS STEFIK: To the best of my knowledge, 00:57:01.480 --> 00:57:03.870 align:middle line:84% the way that Jonathan Godfrey's stuff work is exactly 00:57:03.870 --> 00:57:05.340 align:middle line:90% like you're saying it is. 00:57:05.340 --> 00:57:08.550 align:middle line:84% It's basically like an SVG with some extra, basically, 00:57:08.550 --> 00:57:10.830 align:middle line:84% structure information that gets tossed down. 00:57:10.830 --> 00:57:12.900 align:middle line:84% I can't remember if it's embedded in the SVG 00:57:12.900 --> 00:57:14.430 align:middle line:90% or if it's a separate file. 00:57:14.430 --> 00:57:15.990 align:middle line:90% I forget off the top of my head. 00:57:15.990 --> 00:57:17.820 align:middle line:90% But it's something like that. 00:57:17.820 --> 00:57:21.600 align:middle line:84% The way that we've designed it is we've generated-- 00:57:21.600 --> 00:57:23.730 align:middle line:84% behind the scenes, we generate information 00:57:23.730 --> 00:57:25.710 align:middle line:84% that basically matches to the kind of stuff 00:57:25.710 --> 00:57:27.850 align:middle line:90% that Jonathan puts out. 00:57:27.850 --> 00:57:30.510 align:middle line:84% So hypothetically, you could write a program 00:57:30.510 --> 00:57:33.000 align:middle line:84% that could get that information and then send it off 00:57:33.000 --> 00:57:35.105 align:middle line:90% as an SVG or some other format. 00:57:35.105 --> 00:57:36.480 align:middle line:84% So there's nothing preventing it, 00:57:36.480 --> 00:57:38.725 align:middle line:84% but we haven't done that ourselves. 00:57:38.725 --> 00:57:40.350 align:middle line:84% In part, because we just invented this. 00:57:40.350 --> 00:57:43.620 align:middle line:84% So we could do the operating system of hooks first. 00:57:43.620 --> 00:57:45.510 align:middle line:84% And then however people want to use 00:57:45.510 --> 00:57:48.220 align:middle line:84% it will be kind of based on whatever the community wants, 00:57:48.220 --> 00:57:48.720 align:middle line:90% I guess. 00:57:48.720 --> 00:57:52.050 align:middle line:84% So that stuff is absolutely possible, 100%. 00:57:52.050 --> 00:57:52.800 align:middle line:90% JOOYOUNG: Awesome. 00:57:52.800 --> 00:57:53.633 align:middle line:90% Thank you very much. 00:57:53.633 --> 00:57:55.425 align:middle line:84% ANDREAS STEFIK: There's two other questions 00:57:55.425 --> 00:57:56.527 align:middle line:90% I can answer quickly. 00:57:56.527 --> 00:57:58.110 align:middle line:84% What other charts have you considered? 00:57:58.110 --> 00:58:06.210 align:middle line:84% So far, we've done histograms and bar charts. 00:58:06.210 --> 00:58:08.700 align:middle line:84% We don't use quad trees for either of those, 00:58:08.700 --> 00:58:10.740 align:middle line:84% but my hunch is that quad trees will 00:58:10.740 --> 00:58:13.920 align:middle line:84% be useful to a bunch of things, but we 00:58:13.920 --> 00:58:15.690 align:middle line:90% don't know exactly which ones. 00:58:15.690 --> 00:58:16.440 align:middle line:90% Gary, no problem. 00:58:16.440 --> 00:58:18.570 align:middle line:90% Thanks for coming. 00:58:18.570 --> 00:58:21.960 align:middle line:84% And Lev asks, has anyone used vibration features on phones 00:58:21.960 --> 00:58:24.030 align:middle line:84% to help render graphs in a more accessible way? 00:58:24.030 --> 00:58:25.822 align:middle line:84% Stronger vibrations as you drag your finger 00:58:25.822 --> 00:58:29.350 align:middle line:84% over a larger cluster of points, no vibrations in empty sector? 00:58:29.350 --> 00:58:31.680 align:middle line:84% I think some of the best work that 00:58:31.680 --> 00:58:33.148 align:middle line:84% has been done related to vibration 00:58:33.148 --> 00:58:35.190 align:middle line:84% is done by someone on the call, which is actually 00:58:35.190 --> 00:58:39.180 align:middle line:84% Jenna Gorowitz and also Nick Giudice out of Maine 00:58:39.180 --> 00:58:41.880 align:middle line:84% is a really fantastic scholar on this. 00:58:41.880 --> 00:58:44.610 align:middle line:84% We actually just submitted a NSF proposal 00:58:44.610 --> 00:58:48.390 align:middle line:90% on exactly what you just said. 00:58:48.390 --> 00:58:50.490 align:middle line:84% What was it, a week or two ago, I think, Jenna? 00:58:50.490 --> 00:58:51.820 align:middle line:90% Something like that. 00:58:51.820 --> 00:58:53.520 align:middle line:90% And so we're going to try that. 00:58:53.520 --> 00:58:55.980 align:middle line:84% I suspect that there is some serious potential there, 00:58:55.980 --> 00:58:58.470 align:middle line:84% but it's kind of an open question in the literature, 00:58:58.470 --> 00:58:59.785 align:middle line:90% I think. 00:58:59.785 --> 00:59:01.410 align:middle line:84% Vibration is absolutely useful, it just 00:59:01.410 --> 00:59:03.770 align:middle line:84% depends on for what and under what context. 00:59:03.770 --> 00:59:05.470 align:middle line:90% Yeah, no problem. 00:59:05.470 --> 00:59:06.470 align:middle line:90% BRIANNA BLASER: Awesome. 00:59:06.470 --> 00:59:08.390 align:middle line:84% Well, thank you, Stefik, for the presentation, 00:59:08.390 --> 00:59:10.890 align:middle line:84% and thank you everybody for joining us. 00:59:10.890 --> 00:59:13.250 align:middle line:84% I'll be following up over email and asking 00:59:13.250 --> 00:59:15.650 align:middle line:84% if you want to continue to receive more information 00:59:15.650 --> 00:59:18.150 align:middle line:90% about webinars like this. 00:59:18.150 --> 00:59:19.170 align:middle line:90% Thanks. 00:59:19.170 --> 00:59:21.560 align:middle line:84% ANDREAS STEFIK: Thanks, everybody. 00:59:21.560 --> 00:59:24.000 align:middle line:90%