ScummVM logo Forum Index - ScummVM website - Contact us - Buy Supported Games: GOG.com Rules - Search - Register - Login curved edge
Folder Forum Index > iPhone Port > Implemented Smart Keyboard support for iPad Pro
Implemented Smart Keyboard support for iPad Pro
  Author    Thread Reply to topic
AgustinCordes



Joined: 23 Nov 2011
Posts: 5
Location: Buenos Aires, Argentina
Implemented Smart Keyboard support for iPad Pro 

My very first contribution to ScummVM, and I couldn't be happier! While it's a straightforward implementation, it works very well:

https://youtu.be/I3PZdIpQldI

Basically, I just had to disable the virtual keyboard on landscape orientation, but there are a few extra goodies:

- Removed the annoying iOS shortcuts toolbar for total immersion.
- Added fake ESC key support replacing the ` (thanks @donkthemagicllama for the idea from one of your posts)
- Added full support for arrow keys! This was the trickiest bit to implement but works great.

All of this should theoretically work on any Bluetooth keyboard, but for now I'm only able to test it on iPad Pro w/Smart Keyboard. I'll be pushing my branch soon.

 Reply with quote  
Post Mon Jan 15, 2018 8:52 pm 
 View user's profile Send private message Visit poster's website
digitall
ScummVM Developer


Joined: 02 Aug 2012
Posts: 915
 

AgustinCordes: Thanks for your contribution. Will look forward to seeing a Github Pull Request in future and hopefully it should get merged quickly Smile

 Reply with quote  
Post Tue Jan 16, 2018 12:02 am 
 View user's profile Send private message
dottostring



Joined: 04 Mar 2018
Posts: 4
 

Would you be will ing to share the changes and in what files you made them? I don't see a pull request and use an iPad Pro with Smart Keyboard. Thanks in advance!

 Reply with quote  
Post Tue Mar 06, 2018 12:51 am 
 View user's profile Send private message
dottostring



Joined: 04 Mar 2018
Posts: 4
 

I have decided to share my solution. I couldn't wait!

In file ios7_keyboard.mm:
Change initWithKeyboard to this ->


code:

- (id)initWithKeyboard:(SoftKeyboard *)keyboard {
   self = [super initWithFrame:CGRectMake(0.0f, 0.0f, 0.0f, 0.0f)];
   softKeyboard = keyboard;
   [self setAutocorrectionType:UITextAutocorrectionTypeNo];
   [self setAutocapitalizationType:UITextAutocapitalizationTypeNone];
   [self setEnablesReturnKeyAutomatically:NO];
    UITextInputAssistantItem* item = [self inputAssistantItem];
    item.leadingBarButtonGroups = @[];
    item.trailingBarButtonGroups = @[];
   return self;
}




In file ios7_video.mm:
Change initSurface to this ->


code:

- (void)initSurface {
   if (_context) {
      [self rebuildFrameBuffer];
   }

   BOOL isLandscape = (self.bounds.size.width > self.bounds.size.height); // UIDeviceOrientationIsLandscape([[UIDevice currentDevice] orientation]);

   int screenWidth, screenHeight;
   if (isLandscape) {
      screenWidth = MAX(_renderBufferWidth, _renderBufferHeight);
      screenHeight = MIN(_renderBufferWidth, _renderBufferHeight);
   }
   else {
      screenWidth = MIN(_renderBufferWidth, _renderBufferHeight);
      screenHeight = MAX(_renderBufferWidth, _renderBufferHeight);
   }
   
    if (_keyboardView == nil) {
        _keyboardView = [[SoftKeyboard alloc] initWithFrame:CGRectZero];
        [_keyboardView setInputDelegate:self];
        [self addSubview:[_keyboardView inputView]];
        [self addSubview: _keyboardView];
        [_keyboardView showKeyboard];
    }

   glBindRenderbuffer(GL_RENDERBUFFER, _viewRenderbuffer); printOpenGLError();

   [self clearColorBuffer];

   GLfloat adjustedWidth = _videoContext.screenWidth;
   GLfloat adjustedHeight = _videoContext.screenHeight;
   if (_videoContext.asprectRatioCorrection) {
      if (_videoContext.screenWidth == 320 && _videoContext.screenHeight == 200)
         adjustedHeight = 240;
      else if (_videoContext.screenWidth == 640 && _videoContext.screenHeight == 400)
         adjustedHeight = 480;
   }

   float overlayPortraitRatio;

   if (isLandscape) {
      GLfloat gameScreenRatio = adjustedWidth / adjustedHeight;
      GLfloat screenRatio = (GLfloat)screenWidth / (GLfloat)screenHeight;

      // These are the width/height according to the portrait layout!
      int rectWidth, rectHeight;
      int xOffset, yOffset;

      if (gameScreenRatio < screenRatio) {
         // When the game screen ratio is less than the screen ratio
         // we need to scale the width, since the game screen was higher
         // compared to the width than our output screen is.
         rectWidth = (int)(screenHeight * gameScreenRatio);
         rectHeight = screenHeight;
         xOffset = (screenWidth - rectWidth) / 2;
         yOffset = 0;
      } else {
         // When the game screen ratio is bigger than the screen ratio
         // we need to scale the height, since the game screen was wider
         // compared to the height than our output screen is.
         rectWidth = screenWidth;
         rectHeight = (int)(screenWidth / gameScreenRatio);
         xOffset = 0;
         yOffset = (screenHeight - rectHeight) / 2;
      }

//        [_keyboardView hideKeyboard];

      //printf("Rect: %i, %i, %i, %i\n", xOffset, yOffset, rectWidth, rectHeight);
      _gameScreenRect = CGRectMake(xOffset, yOffset, rectWidth, rectHeight);
      overlayPortraitRatio = 1.0f;
   } else {
      GLfloat ratio = adjustedHeight / adjustedWidth;
      int height = (int)(screenWidth * ratio);
      //printf("Making rect (%u, %u)\n", screenWidth, height);
      _gameScreenRect = CGRectMake(0, 0, screenWidth, height);

//        CGRect keyFrame = CGRectMake(0.0f, 0.0f, 0.0f, 0.0f);
//        if (_keyboardView == nil) {
//            _keyboardView = [[SoftKeyboard alloc] initWithFrame:keyFrame];
//            [_keyboardView setInputDelegate:self];
//            [self addSubview:[_keyboardView inputView]];
//            [self addSubview: _keyboardView];
//        }
//
//        [_keyboardView showKeyboard];
      overlayPortraitRatio = (_videoContext.overlayHeight * ratio) / _videoContext.overlayWidth;
   }
   _overlayRect = CGRectMake(0, 0, screenWidth, screenHeight * overlayPortraitRatio);

   _gameScreenCoords[0].x = _gameScreenCoords[2].x = CGRectGetMinX(_gameScreenRect);
   _gameScreenCoords[0].y = _gameScreenCoords[1].y = CGRectGetMinY(_gameScreenRect);
   _gameScreenCoords[1].x = _gameScreenCoords[3].x = CGRectGetMaxX(_gameScreenRect);
   _gameScreenCoords[2].y = _gameScreenCoords[3].y = CGRectGetMaxY(_gameScreenRect);

   _overlayCoords[1].x = _overlayCoords[3].x = CGRectGetMaxX(_overlayRect);
   _overlayCoords[2].y = _overlayCoords[3].y = CGRectGetMaxY(_overlayRect);

   [self setViewTransformation];
   [self updateMouseCursorScaling];
}




Then in the same file change handleKeyPress to this ->

code:

- (void)handleKeyPress:(unichar)c {
    if (c == '`') {
        [self addEvent:InternalEvent(kInputKeyPressed, '\E', 0)];
    } else {
        [self addEvent:InternalEvent(kInputKeyPressed, c, 0)];
    }
}




Finally:
Enjoy Space Quest!
 Reply with quote  
Post Sun Mar 11, 2018 7:16 pm 
 View user's profile Send private message
dottostring



Joined: 04 Mar 2018
Posts: 4
 

Directly under SofKeyboard init function, add -->

code:

- (NSArray *)keyCommands {
    UIKeyCommand *upArrow = [UIKeyCommand keyCommandWithInput: UIKeyInputUpArrow modifierFlags: 0 action: @selector(upArrow:)];
    UIKeyCommand *downArrow = [UIKeyCommand keyCommandWithInput: UIKeyInputDownArrow modifierFlags: 0 action: @selector(downArrow:)];
    UIKeyCommand *leftArrow = [UIKeyCommand keyCommandWithInput: UIKeyInputLeftArrow modifierFlags: 0 action: @selector(leftArrow:)];
    UIKeyCommand *rightArrow = [UIKeyCommand keyCommandWithInput: UIKeyInputRightArrow modifierFlags: 0 action: @selector(rightArrow:)];
    return [[NSArray alloc] initWithObjects: upArrow, downArrow, leftArrow, rightArrow, nil];
}

- (void) upArrow: (UIKeyCommand *) keyCommand {
//    [self resignFirstResponder];
    [softKeyboard handleKeyPress:0x4800];
}

- (void) downArrow: (UIKeyCommand *) keyCommand {
    [softKeyboard handleKeyPress:0x5000];
}

- (void) leftArrow: (UIKeyCommand *) keyCommand {
    [softKeyboard handleKeyPress:0x4B00];
}

- (void) rightArrow: (UIKeyCommand *) keyCommand {
    [softKeyboard handleKeyPress:0x4D00];
}




This supports the keyboard arrow!
 Reply with quote  
Post Sun Mar 11, 2018 8:21 pm 
 View user's profile Send private message
dottostring



Joined: 04 Mar 2018
Posts: 4
 

I apologize, replace the above with this for better functionality between games it appears:

- (void) upArrow: (UIKeyCommand *) keyCommand {
// [self resignFirstResponder];
[softKeyboard handleKeyPress:273];
}

- (void) downArrow: (UIKeyCommand *) keyCommand {
[softKeyboard handleKeyPress:274];
}

- (void) leftArrow: (UIKeyCommand *) keyCommand {
[softKeyboard handleKeyPress:276];
}

- (void) rightArrow: (UIKeyCommand *) keyCommand {
[softKeyboard handleKeyPress:275];
}

 Reply with quote  
Post Sun Mar 11, 2018 8:47 pm 
 View user's profile Send private message
basis



Joined: 27 May 2017
Posts: 1
 

Thank you so much!!!

Any chance we can get this added in to the core code so that when an update comes out I do not have to manually change this?

Thanks again.

 Reply with quote  
Post Sat Jun 23, 2018 8:39 pm 
 View user's profile Send private message
digitall
ScummVM Developer


Joined: 02 Aug 2012
Posts: 915
 

I have applied the patches from these posts with minor formatting cleanup, and opened this as a Pull Request as:
https://github.com/scummvm/scummvm/pull/1225

@dottostring: If you wish to check my patch / compile / test or comment there, feel free. If you want me to amend the commit to have your github attribution, let me know.

 Reply with quote  
Post Sun Jun 24, 2018 1:45 am 
 View user's profile Send private message
MWumpusZ



Joined: 17 Aug 2018
Posts: 5
 

I've been trying this out, and had a number of issues.


Arrow keys:

As committed and merged, arrow keys did not work for me. Possibly there was a misunderstanding about where the keyCommands function needed to be added? In any case, I moved keyCommands to iPhoneView and it works that way, PR is here: https://github.com/scummvm/scummvm/pull/1290


Virtual keyboard:

After the change, the virtual keyboard never appears, as far as I can tell. Possibly this was intentional, but surely its not a generally acceptable situation - making an external keyboard NECESSARY to play games seems like a bad idea. I haven't looked into fixing this yet.


Stops responding to keyboard input:

"Sometimes" typing doesn't end up in the games input fields. My bluetooth keyboard has a button for searching my ipad, and after pressing this and returning to scummvm, I can not enter text. Certain details strongly suggest that it's not an issue with my keyboard, and I have a few leads, but will be investigating further.


Unclear UITextAssistant stuff:

Apparently to make the build work, digitall disabled code that does... something... with a UITextInputAssistantItem, see https://github.com/scummvm/scummvm/commit/f353e466df2175f10e4328bce3f6a44e295ba305. It's not clear to me what this disabled code was supposed to do; for me, reenabling it appears to build just fine, but no effect is apparent - what is it supposed to do?


It sounds like a whole bunch of complaints, but it's great work that has been done by you guys! It looks like fixing them up shouldn't be that hard. I'll certainly be looking into various points, but any thoughts and observations - including from anyone else who has tried the changes out - would be great.

 Reply with quote  
Post Fri Aug 17, 2018 10:29 am 
 View user's profile Send private message
MWumpusZ



Joined: 17 Aug 2018
Posts: 5
 

Incidentally, do you know how input with physical external keyboards, and virtual keyboard, and so on, is done in the android version? I have an android tablet I could try it on, but no android development environment set up Neutral

My reason for asking: It MAY be wise to have the same methods of user interaction for both android and ios, insofar as it is reasonable to do so.

 Reply with quote  
Post Fri Aug 17, 2018 10:33 am 
 View user's profile Send private message
MWumpusZ



Joined: 17 Aug 2018
Posts: 5
 

quote:
Originally posted by MWumpusZ
Stops responding to keyboard input:

"Sometimes" typing doesn't end up in the games input fields. My bluetooth keyboard has a button for searching my ipad, and after pressing this and returning to scummvm, I can not enter text. Certain details strongly suggest that it's not an issue with my keyboard, and I have a few leads, but will be investigating further.


It turns out this is not entirely new; I reverted to before the merge and was able to reproduce this there:

* Connect bluetooth keyboard
* Turn to portrait mode
-> virtual keyboard DOES NOT appear (this is OK, its just an observation)
-> bluetooth keyboard input is now "accepted"
* Press search ipad button
* Cancel
-> bluetooth keyboard input is NOT accepted

Rotating to landscape and back makes it works again, so evidently that action "resets" something relevant.

In any case, this is evidently a separate bug.
 Reply with quote  
Post Fri Aug 17, 2018 3:13 pm 
 View user's profile Send private message
MWumpusZ



Joined: 17 Aug 2018
Posts: 5
 

quote:
Originally posted by MWumpusZ
Unclear UITextAssistant stuff:

Apparently to make the build work, digitall disabled code that does... something... with a UITextInputAssistantItem, see https://github.com/scummvm/scummvm/commit/f353e466df2175f10e4328bce3f6a44e295ba305. It's not clear to me what this disabled code was supposed to do; for me, reenabling it appears to build just fine, but no effect is apparent - what is it supposed to do?


Evidently it hides the "minimised" keyboard bar that appears when the virtual keyboard is hidden. For some reason on my physical 2018 iPad (6th generation) this does not appear regardless of whether or not this code is included, but when simulating the same device it does, and adding/removing this code has the expected effect. Wut?
 Reply with quote  
Post Sat Aug 18, 2018 5:20 am 
 View user's profile Send private message
digitall
ScummVM Developer


Joined: 02 Aug 2012
Posts: 915
 

MWumpusZ : Thanks for taking a look at this code and seeing about fixing it up. I am not an iOS expert and don't even own any Apple or iOS devices, so could not test this, but though it best to get the code into the tree and worked on. I tried to fix it up as far as possible where logical, but I am afraid I am not an expert here so can't offer more guidance / knowledge.

 Reply with quote  
Post Mon Aug 20, 2018 7:08 pm 
 View user's profile Send private message
MWumpusZ



Joined: 17 Aug 2018
Posts: 5
 

quote:
Originally posted by MWumpusZ
quote:
Originally posted by MWumpusZ
Unclear UITextAssistant stuff:


Evidently it hides the "minimised" keyboard bar that appears when the virtual keyboard is hidden. For some reason on my physical 2018 iPad (6th generation) this does not appear regardless of whether or not this code is included, but when simulating the same device it does ...


Whether or not this bar appears at all is evidently (also) influenced by the keyboard configuration in the system settings. Fun!
 Reply with quote  
Post Mon Aug 20, 2018 10:01 pm 
 View user's profile Send private message
  Display posts from previous:      
Reply to topic

Forum Jump:
 



Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Powered by phpBB © 2001, 2006 phpBB Group
Forum design by ScummVM team, icons by raina
curved edge   curved edge